[
https://issues.apache.org/jira/browse/SPARK-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721782#comment-15721782
]
Apache Spark commented on SPARK-18716:
--------------------------------------
User 'uncleGen' has created a pull request for this issue:
https://github.com/apache/spark/pull/16142
> Restrict the disk usage of spark event log.
> --------------------------------------------
>
> Key: SPARK-18716
> URL: https://issues.apache.org/jira/browse/SPARK-18716
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.2
> Reporter: Genmao Yu
>
> We've had reports of overfull disk usage of spark event log file. Current
> implementation has following drawbacks:
> 1. If we did not start Spark HistoryServer or Spark HistoryServer just
> failed, there is no chance to do clean work.
> 2. Spark HistoryServer is cleaning event log file based on file age only. If
> there are abundant applications constantly, the disk usage in every
> {{spark.history.fs.cleaner.maxAge}} can still be very large.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
