[ https://issues.apache.org/jira/browse/SPARK-21223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064406#comment-16064406 ]
Apache Spark commented on SPARK-21223: -------------------------------------- User 'zenglinxi0615' has created a pull request for this issue: https://github.com/apache/spark/pull/18430 > Thread-safety issue in FsHistoryProvider > ----------------------------------------- > > Key: SPARK-21223 > URL: https://issues.apache.org/jira/browse/SPARK-21223 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.1.1 > Reporter: zenglinxi > > Currently, Spark HistoryServer use a HashMap named fileToAppInfo in class > FsHistoryProvider to store the map of eventlog path and attemptInfo. > When use ThreadPool to Replay the log files in the list and merge the list of > old applications with new ones, multi thread may update fileToAppInfo at the > same time, which may cause Thread-safety issues. > {code:java} > for (file <- logInfos) { > tasks += replayExecutor.submit(new Runnable { > override def run(): Unit = mergeApplicationListing(file) > }) > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org