[
https://issues.apache.org/jira/browse/SPARK-43861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-43861:
-----------------------------------
Labels: pull-request-available (was: )
> History Server delete event log of running Spark applicaiton
> ------------------------------------------------------------
>
> Key: SPARK-43861
> URL: https://issues.apache.org/jira/browse/SPARK-43861
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.4.0
> Reporter: shengkui leng
> Priority: Minor
> Labels: pull-request-available
>
> When enable Spark history cleaner:
> {noformat}
> spark.history.fs.cleaner.enabled=true
> spark.history.fs.cleaner.interval=1d
> spark.history.fs.cleaner.maxAge=7d
> spark.history.fs.cleaner.maxNum=10
> {noformat}
> Spark will delete history for running Spark applications, following is some
> message picked from the spark log:
> {noformat}
> 153 23/05/26 14:19:43 INFO FsHistoryProvider: Parsing
> /tmp/spark-events/local-1685081895370 to re-build UI...
> 154 23/05/26 14:19:43 INFO FsHistoryProvider: Finished parsing
> /tmp/spark-events/local-1685081895370
> 155 23/05/28 13:09:53 INFO FsHistoryProvider: Deleting expired event log for
> app-20230524141002-0000
> 156 23/05/29 13:09:53 INFO FsHistoryProvider: Deleting expired event log for
> local-1684999095807
> 157 23/05/29 13:09:53 INFO FsHistoryProvider: Deleting expired event log for
> app-20230524171730-0000.inprogress
> 158 23/05/29 13:34:24 INFO ApplicationCache: Failed to load application
> attempt app-20230524171730-0000/
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
