[jira] [Created] (SPARK-2668) Support log4j log to yarn container log directory

[Peng Zhang (JIRA)]

Peng Zhang created SPARK-2668:
---------------------------------

             Summary: Support log4j log to yarn container log directory
                 Key: SPARK-2668
                 URL: https://issues.apache.org/jira/browse/SPARK-2668
             Project: Spark
          Issue Type: Improvement
          Components: YARN
            Reporter: Peng Zhang
             Fix For: 1.0.0


Assign value of yarn container log directory to java opts "spark.yarn.log.dir", 
So user defined log4j.properties can reference this value and write log to YARN 
container directory.
Otherwise, user defined file append will log to CWD, and files will not be 
displayed on YARN UI，and either cannot be aggregated to HDFS log directory 
after job finished.

User defined log4j.properties reference example:
{code}
log4j.appender.rolling_file.File = ${spark.yarn.log.dir}/spark.log
{code}





--
This message was sent by Atlassian JIRA
(v6.2#6252)