liujianhui created SPARK-18225:
----------------------------------
Summary: job will miss when driver removed by master in spark
streaming
Key: SPARK-18225
URL: https://issues.apache.org/jira/browse/SPARK-18225
Project: Spark
Issue Type: Bug
Components: DStreams, Scheduler
Affects Versions: 1.6.2, 1.6.1
Reporter: liujianhui
kill the application on spark ui, the master will send an ApplicationRemoved to
driver, driver will abort the all pending job,and then the job finish with
exception "Master removed our application:Killed",and then Jobscheduler will
remove the job from jobsets, but the jobgenerator still docheckpoint without
the job which removed before, and then driver stop；when recover from the check
point file，it miss all jobs which aborted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
