[ https://issues.apache.org/jira/browse/SPARK-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996013#comment-15996013 ]
zhoukang edited comment on SPARK-20582 at 5/4/17 1:44 AM: ---------------------------------------------------------- You are right [~vanzin],i have looked into you implementation.Thanks for your time. was (Author: cane): You are right [~vanzin],i searched into you implementation.Thanks for your time. > Speed up the restart of HistoryServer using ApplicationAttemptInfo > checkpointing > -------------------------------------------------------------------------------- > > Key: SPARK-20582 > URL: https://issues.apache.org/jira/browse/SPARK-20582 > Project: Spark > Issue Type: Improvement > Components: Deploy > Affects Versions: 2.1.0 > Reporter: zhoukang > Priority: Critical > > In the current code of HistoryServer,jetty server will be started > after all the logs fetch from yarn has been replayed.However,when the number > of logs is becoming larger,the start time of jetty will be too long. > Here,we implement a solution which using ApplicationAttemptInfo > checkpointing to speed up the start of historyserver.When historyserver is > starting,it will load ApplicationAttemptInfo from checkpoint file first if > exists which is faster then replaying one by one. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org