[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770413#comment-13770413
 ] 

Zhijie Shen commented on MAPREDUCE-5505:
----------------------------------------

bq. Ideally we should have the state transition itself to go to SUCCEEDED after 
unregister.

unregister needs to happens when a job is after COMMITTING. If we want 
unregister to be before SUCCEEDED, we have to insert another state in middle, 
where unregister can happen. Then, we still need to to map the state in middle 
to a external state. If we map it to SUCCEEDED, the problem is still there; if 
we map it to RUNNING, it is equivalent to returning RUNNING when AM is 
unregistered (in the prior comment).

In addition, the problem is not limited to SUCCEEDED only, is it? FAILED, 
KILLED and ERROR shouldn't be returned before unregister. In particular, FAILED 
and KILLED map to two internal states respectively. In this case, I'm afraid 
it's even more difficult to find state in between to execute unregister.

bq. 1) JobImpl go to succeeded/terminal state only after app master 
serviceStop() has completed?

In addition to the reason above, if serviceStop() is completed, the client is 
already unable to get the latest state from AM.

bq. 2) Change the MRClientProtocol to return job.state as the previous job 
state when job.state == succeeded. Then when MRClientProtocol.stop() is called 
it starts returning job.state == succeeded for 5 secs. Then it stops and the 
app master stops.

+1 for this method. It sounds work with limited touch. Again, we should 
consider FAILED, KILLED and ERROR as well, right?
                
> Clients should be notified job finished only after job successfully 
> unregistered 
> ---------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5505
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5505
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Jian He
>            Assignee: Zhijie Shen
>
> This is to make sure user is notified job finished after job is really done. 
> This does increase client latency but can reduce some races during unregister 
> like YARN-540

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to