[
https://issues.apache.org/jira/browse/SPARK-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15761941#comment-15761941
]
Marcelo Vanzin commented on SPARK-18716:
----------------------------------------
For posterity, another problem with this feature that I didn't mention in the
PR, is that allow users to use the SHS to delete content created by other
users. A malicious user can just write a big event log file in the SHS
directory and that would eventually make the SHS delete log files from other
users.
> Restrict the disk usage of spark event log.
> --------------------------------------------
>
> Key: SPARK-18716
> URL: https://issues.apache.org/jira/browse/SPARK-18716
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.2
> Reporter: Genmao Yu
>
> We've had reports of overfull disk usage of spark event log file. Current
> implementation has following drawbacks:
> 1. If we did not start Spark HistoryServer or Spark HistoryServer just
> failed, there is no chance to do clean work.
> 2. Spark HistoryServer is cleaning event log file based on file age only. If
> there are abundant applications constantly, the disk usage in every
> {{spark.history.fs.cleaner.maxAge}} can still be very large.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
