[ https://issues.apache.org/jira/browse/MAPREDUCE-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617366#comment-13617366 ]
Arun C Murthy commented on MAPREDUCE-4824: ------------------------------------------ Tom - sorry, looks like we lost track of this. Is this good to go? Tx > Provide a mechanism for jobs to indicate they should not be recovered on > restart > -------------------------------------------------------------------------------- > > Key: MAPREDUCE-4824 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4824 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: mrv1 > Affects Versions: 1.1.0 > Reporter: Tom White > Assignee: Tom White > Attachments: MAPREDUCE-4824.patch, MAPREDUCE-4824.patch, > MAPREDUCE-4824.patch, MAPREDUCE-4824.patch, MAPREDUCE-4824.patch, > MAPREDUCE-4824.patch > > > Some jobs (like Sqoop or HBase jobs) are not idempotent, so should not be > recovered on jobtracker restart. MAPREDUCE-2702 solves this problem for MR2, > however the approach there is not applicable for MR1, since even if we only > use the job-level part of the patch and add a isRecoverySupported method to > OutputCommitter, there is no way to use that information from the JT (which > initiates recovery), since the JT does not instantiate OutputCommitters - and > it shouldn't since they are user-level code. (In MR2 it's OK since the MR AM > calls the method.) > Instead, we can add a MR configuration property to say that a job is not > recoverable, and the JT could safely read this from the job conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira