Sorted. The new section in log4j.properties has to be modified
# Set the default spark-shell log level to WARN. When running the
spark-shell, the
# log level for this class is used to overwrite the root logger's log
level, so that
# the user can have different defaults for the shell and regular
Hi,
Just doing some tests on Spark 2 version spark-shell
using spark-shell, in my $SPARK_HOME/conf/log4j.properties, I have:
log4j.rootCategory=ERROR, console
I get
spark-shell
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
I don't have this issue
