[ https://issues.apache.org/jira/browse/SPARK-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995958#comment-15995958 ]
Marcelo Vanzin commented on SPARK-20582: ---------------------------------------- You're talking about checkpointing the application info, which the code I have already does. With my changes the HTTP server comes up immediately. > Speed up the restart of HistoryServer using ApplicationAttemptInfo > checkpointing > -------------------------------------------------------------------------------- > > Key: SPARK-20582 > URL: https://issues.apache.org/jira/browse/SPARK-20582 > Project: Spark > Issue Type: Improvement > Components: Deploy > Affects Versions: 2.1.0 > Reporter: zhoukang > Priority: Critical > > In the current code of HistoryServer,jetty server will be started > after all the logs fetch from yarn has been replayed.However,when the number > of logs is becoming larger,the start time of jetty will be too long. > Here,we implement a solution which using ApplicationAttemptInfo > checkpointing to speed up the start of historyserver.When historyserver is > starting,it will load ApplicationAttemptInfo from checkpoint file first if > exists which is faster then replaying one by one. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org