Re: spark streaming: issue with logging with separate log4j properties files for driver and executor

Resolved. Used passing parameters in sparkConf instead of passing to spark-submit command : (still dont know why passing to spark-submit command did not work) sparkConf.set("spark.executor.extraJavaOptions", "-XX:+UseG1GC -Dlog4j.configuration=file:log4j_RequestLogExecutor.properties ") On

Re: spark streaming: issue with logging with separate log4j properties files for driver and executor

Any suggestion? On Mon, May 23, 2016 at 5:18 PM, chandan prakash wrote: > Hi, > I am able to do logging for driver but not for executor. > > I am running spark streaming under mesos. > Want to do log4j logging separately for driver and executor. > > Used the below

spark streaming: issue with logging with separate log4j properties files for driver and executor

Hi, I am able to do logging for driver but not for executor. I am running spark streaming under mesos. Want to do log4j logging separately for driver and executor. Used the below option in spark-submit command : --driver-java-options