Here’s an example. I echoed JAVA_OPTS so that you can see what I’ve got. Then I call ‘activator run’ in the project directory.
jjones-mac:analyzer-perf jjones$ echo $JAVA_OPTS -Xmx4g -Xmx4g -Dlog4j.configuration=file:/Users/jjones/src/adaptive/adaptiveobjects/analyzer-perf/conf/log4j.properties jjones-mac:analyzer-perf jjones$ activator run [info] Loading project definition from /Users/jjones/src/adaptive/adaptiveobjects/analyzer-perf/project [info] Set current project to analyzer-perf (in build file:/Users/jjones/src/adaptive/adaptiveobjects/analyzer-perf/) [info] Running com.adaptive.analyzer.perf.AnalyzerPerf 11:15:24.066 [run-main-0] INFO org.apache.spark.SparkContext - Running Spark version 1.4.1 11:15:24.150 [run-main-0] DEBUG o.a.h.m.lib.MutableMetricsFactory - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, value=[Rate of successful kerberos logins and latency (milliseconds)], valueName=Time) 11:15:24.156 [run-main-0] DEBUG o.a.h.m.lib.MutableMetricsFactory - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, value=[Rate of failed kerberos logins and latency (milliseconds)], valueName=Time) As I mentioned below but repeated for completeness, I also have this in my code. import org.apache.log4j.PropertyConfigurator PropertyConfigurator.configure("conf/log4j.properties") Logger.getRootLogger().setLevel(Level.OFF) Logger.getLogger("org").setLevel(Level.OFF) Logger.getLogger("akka").setLevel(Level.OFF) And here’s my log4j.properties (note, I’ve also tried setting the level to OFF): # Set everything to be logged to the console log4j.rootCategory=WARN log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n # Change this to set Spark log level log4j.logger.org.apache.spark=WARN # Silence akka remoting log4j.logger.Remoting=WARN # Ignore messages below warning level from Jetty, because it's a bit verbose log4j.logger.org.eclipse.jetty=WARN spark.log.threshold=OFF spark.root.logger=OFF,DRFA From: Alex Kozlov Date: Tuesday, October 6, 2015 at 10:50 AM To: Jeff Jones Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Re: How can I disable logging when running local[*]? Try JAVA_OPTS='-Dlog4j.configuration=file:/<path-to-log4j.properties>' Internally, this is just spark.driver.extraJavaOptions, which you should be able to set in conf/spark-defaults.conf Can you provide more details how you invoke the driver? On Tue, Oct 6, 2015 at 9:48 AM, Jeff Jones <jjo...@adaptivebiotech.com<mailto:jjo...@adaptivebiotech.com>> wrote: Thanks. Any chance you know how to pass this to a Scala app that is run via TypeSafe activator? I tried putting it $JAVA_OPTS but I get: Unrecognized option: --driver-java-options Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. I tried a bunch of different quoting but nothing produced a good result. I also tried passing it directly to activator using –jvm but it still produces the same results with verbose logging. Is there a way I can tell if it’s picking up my file? From: Alex Kozlov Date: Monday, October 5, 2015 at 8:34 PM To: Jeff Jones Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Re: How can I disable logging when running local[*]? Did you try “--driver-java-options '-Dlog4j.configuration=file:/<path-to-log4j.properties>'” and setting the log4j.rootLogger=FATAL,console? On Mon, Oct 5, 2015 at 8:19 PM, Jeff Jones <jjo...@adaptivebiotech.com<mailto:jjo...@adaptivebiotech.com>> wrote: I’ve written an application that hosts the Spark driver in-process using “local[*]”. I’ve turned off logging in my conf/log4j.properties file. I’ve also tried putting the following code prior to creating my SparkContext. These were coupled together from various posts I’ve. None of these steps have worked. I’m still getting a ton of logging to the console. Anything else I can try? Thanks, Jeff private def disableLogging(): Unit = { import org.apache.log4j.PropertyConfigurator PropertyConfigurator.configure("conf/log4j.properties") Logger.getRootLogger().setLevel(Level.OFF) Logger.getLogger("org").setLevel(Level.OFF) Logger.getLogger("akka").setLevel(Level.OFF) } This message (and any attachments) is intended only for the designated recipient(s). It may contain confidential or proprietary information, or have other limitations on use as indicated by the sender. If you are not a designated recipient, you may not review, use, copy or distribute this message. If you received this in error, please notify the sender by reply e-mail and delete this message. -- Alex Kozlov (408) 507-4987<tel:%28408%29%20507-4987> (408) 830-9982<tel:%28408%29%20830-9982> fax (650) 887-2135<tel:%28650%29%20887-2135> efax ale...@gmail.com<mailto:ale...@gmail.com> This message (and any attachments) is intended only for the designated recipient(s). It may contain confidential or proprietary information, or have other limitations on use as indicated by the sender. If you are not a designated recipient, you may not review, use, copy or distribute this message. If you received this in error, please notify the sender by reply e-mail and delete this message. This message (and any attachments) is intended only for the designated recipient(s). It may contain confidential or proprietary information, or have other limitations on use as indicated by the sender. If you are not a designated recipient, you may not review, use, copy or distribute this message. If you received this in error, please notify the sender by reply e-mail and delete this message.