[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964471#comment-15964471 ]
Josh Bacon commented on SPARK-16784: ------------------------------------ [~tscholak] I have not created a follow up issue for this. For your situation I'd suggest trying --driver-java-options='..' instead of --conf='spark.driver.extraJavaOptions=...' because the latter is applied after driver jvm actually starts (too late for log4j). My use-case I abandoned attempting to configured log4j for executors, but was able to work with driver/application logs in both cluster and client mode (standalone) via baking my log4j.properties files into my apps Uber jar resources. I think the need of this issue is a new feature for distributing files/log4j.properties in the cluster before the actual Spark Driver starts which I'd imagine is not a pressing enough to warrant actual development at the moment. > Configurable log4j settings > --------------------------- > > Key: SPARK-16784 > URL: https://issues.apache.org/jira/browse/SPARK-16784 > Project: Spark > Issue Type: Improvement > Affects Versions: 2.0.0, 2.1.0 > Reporter: Michael Gummelt > > I often want to change the logging configuration on a single spark job. This > is easy in client mode. I just modify log4j.properties. It's difficult in > cluster mode, because I need to modify the log4j.properties in the > distribution in which the driver runs. I'd like a way of setting this > dynamically, such as a java system property. Some brief searching showed > that log4j doesn't seem to accept such a property, but I'd like to open up > this idea for further comment. Maybe we can find a solution. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org