zenglinxi created SPARK-21223:
---------------------------------
Summary: Thread-safety issue in FsHistoryProvider
Key: SPARK-21223
URL: https://issues.apache.org/jira/browse/SPARK-21223
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.1.1
Reporter: zenglinxi
Currently, Spark HistoryServer use a HashMap named fileToAppInfo in class
FsHistoryProvider to store the map of eventlog path and attemptInfo.
When use ThreadPool to Replay the log files in the list and merge the list of
old applications with new ones, multi thread may update fileToAppInfo at the
same time, which may cause Thread-safety issues.
{code:java}
for (file <- logInfos) {
tasks += replayExecutor.submit(new Runnable {
override def run(): Unit = mergeApplicationListing(file)
})
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
