Saisai Shao created SPARK-15990:
-----------------------------------
Summary: Support rolling log aggregation for Spark running on YARN
Key: SPARK-15990
URL: https://issues.apache.org/jira/browse/SPARK-15990
Project: Spark
Issue Type: Improvement
Components: YARN
Reporter: Saisai Shao
Priority: Minor
YARN supports rolling log aggregation since version 2.6+, it will aggregate the
logs in a timely manner and upload to the HDFS. Compared to the previous log
aggregation method which only aggregate the logs after application is finished,
this will speed up the log aggregation time. Also this will avoid too large log
file problem (out of disk).
So here propose to introduce this feature for Spark on YARN.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
