[
https://issues.apache.org/jira/browse/SPARK-18225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-18225.
----------------------------------
Resolution: Incomplete
> job will miss when driver removed by master in spark streaming
> ---------------------------------------------------------------
>
> Key: SPARK-18225
> URL: https://issues.apache.org/jira/browse/SPARK-18225
> Project: Spark
> Issue Type: Bug
> Components: DStreams, Scheduler
> Affects Versions: 1.6.1, 1.6.2
> Reporter: liujianhui
> Priority: Major
> Labels: bulk-closed
>
> kill the application on spark ui, the master will send an ApplicationRemoved
> to driver, driver will abort the all pending job,and then the job finish with
> exception "Master removed our application:Killed",and then Jobscheduler will
> remove the job from jobsets, but the jobgenerator still docheckpoint without
> the job which removed before, and then driver stop；when recover from the
> check point file，it miss all jobs which aborted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
