shengkui leng created SPARK-43861:
-------------------------------------
Summary: History Server delete event log of running Spark
applicaiton
Key: SPARK-43861
URL: https://issues.apache.org/jira/browse/SPARK-43861
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.4.0
Reporter: shengkui leng
When enable Spark history cleaner:
{noformat}
spark.history.fs.cleaner.enabled=true
spark.history.fs.cleaner.interval=1d
spark.history.fs.cleaner.maxAge=7d
spark.history.fs.cleaner.maxNum=10
{noformat}
Spark will delete history for running Spark applications, following is some
message picked from the spark log:
{noformat}
153 23/05/26 14:19:43 INFO FsHistoryProvider: Parsing
/tmp/spark-events/local-1685081895370 to re-build UI...
154 23/05/26 14:19:43 INFO FsHistoryProvider: Finished parsing
/tmp/spark-events/local-1685081895370
155 23/05/28 13:09:53 INFO FsHistoryProvider: Deleting expired event log for
app-20230524141002-0000
156 23/05/29 13:09:53 INFO FsHistoryProvider: Deleting expired event log for
local-1684999095807
157 23/05/29 13:09:53 INFO FsHistoryProvider: Deleting expired event log for
app-20230524171730-0000.inprogress
158 23/05/29 13:34:24 INFO ApplicationCache: Failed to load application attempt
app-20230524171730-0000/
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
