[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peng Zhang updated SPARK-2668: ------------------------------ Summary: Add variable of yarn log directory for reference from the log4j configuration (was: Add variable of yarn log directory to reference from the log4j configuration) > Add variable of yarn log directory for reference from the log4j configuration > ----------------------------------------------------------------------------- > > Key: SPARK-2668 > URL: https://issues.apache.org/jira/browse/SPARK-2668 > Project: Spark > Issue Type: Improvement > Components: YARN > Affects Versions: 1.0.0 > Reporter: Peng Zhang > > Assign value of yarn container log directory to java opts > "spark.yarn.log.dir", So user defined log4j.properties can reference this > value and write log to YARN container directory. > Otherwise, user defined file appender will only write to container's CWD, and > log files in CWD will not be displayed on YARN UIļ¼and either cannot be > aggregated to HDFS log directory after job finished. > User defined log4j.properties reference example: > {code} > log4j.appender.rolling_file.File = ${spark.yarn.log.dir}/spark.log > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)