Edit your conf/log4j.properties file and Change the following line:
log4j.rootCategory=INFO, console
to
log4j.rootCategory=ERROR, console
Another approach would be to :
Fireup spark-shell and type in the following:
import org.apache.log4j.Logger
import
Another option is to make a copy of log4j.properties in the current
directory where you start spark-shell from, and modify
log4j.rootCategory=INFO,
console to log4j.rootCategory=ERROR, console. Then start the shell.
On Wed, Jan 7, 2015 at 3:39 AM, Akhil ak...@sigmoidanalytics.com wrote:
Edit
go to your spark home and then into the conf/ directory and then edit the
log4j.properties file i.e. :
gedit $SPARK_HOME/conf/log4j.properties
and set root logger to:
log4j.rootCategory=WARN, console
U don't need to build spark for the changes to take place. Whenever you
open spark-shel, it
Tried --driver-java-options and SPARK_JAVA_OPTS, none of them worked
Had to change the default one and rebuilt.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/disable-log4j-for-spark-shell-tp11278p18513.html
Sent from the Apache Spark User List mailing
Even after changing
core/src/main/resources/org/apache/spark/log4j-defaults.properties to WARN
followed by a rebuild, the log level is still INFO.
Any other suggestions?
--
View this message in context:
Some console messages:
14/11/10 20:04:33 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:46713
14/11/10 20:04:33 INFO util.Utils: Successfully started service 'HTTP file
server' on port 46713.
14/11/10 20:04:34 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/11/10 20:04:34 INFO
public static void main(String[] args) throws Exception {
System.out.println(Set Log to Warn);
Logger rootLogger = Logger.getRootLogger();
rootLogger.setLevel(Level.WARN);
...
works for me
--
View this message in context:
You just have to tell Spark which log4j properties file to use. I think
--driver-java-options=-Dlog4j.configuration=log4j.properties should work
but it didn't for me. set
SPARK_JAVA_OPTS=-Dlog4j.configuration=log4j.properties did work though (this
was on Windows, in local mode, assuming you put a
If someone doesn't have the access to do that is there any easy to specify a
different properties file to be used?
Patrick Wendell wrote
If you want to customize the logging behavior - the simplest way is to
copy
conf/log4j.properties.tempate to conf/log4j.properties. Then you can go
and
That's just a template. Nothing consults that file by default.
It's looking inside the Spark .jar. If you edit
core/src/main/resources/org/apache/spark/log4j-defaults.properties and
rebuild Spark, it will pick up those changes.
I think you could also use the JVM argument
If you want to customize the logging behavior - the simplest way is to copy
conf/log4j.properties.tempate to conf/log4j.properties. Then you can go and
modify the log level in there. The spark shells should pick this up.
On Sun, Aug 3, 2014 at 6:16 AM, Sean Owen so...@cloudera.com wrote:
11 matches
Mail list logo
