[ https://issues.apache.org/jira/browse/MAPREDUCE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mohammad Kamrul Islam updated MAPREDUCE-5812: --------------------------------------------- Attachment: MAPREDUCE-5812.1.patch Patch uploaded based on option 2. Testing done: * New test cases added. Also tested in Hive with an outstanding RM restart patch in a single node cluster. > Make task context available to OutputCommitter.isRecoverySupported() > --------------------------------------------------------------------- > > Key: MAPREDUCE-5812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5812 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am > Affects Versions: 2.3.0 > Reporter: Mohammad Kamrul Islam > Assignee: Mohammad Kamrul Islam > Attachments: MAPREDUCE-5812.1.patch > > > Background > ========== > The system like Hive provides its version of OutputCommitter. The custom > implementation of isRecoverySupported() requires task context. From > taskContext:getConfiguration(), hive checks if hive-defined specific > property is set or not. Based on the property value, it returns true or > false. However, in the current OutputCommitter:isRecoverySupported(), there > is no way of getting task config. As a result, user can't turn on/off the > MRAM recovery feature. > Proposed resolution: > =============== > 1. Pass Task Context into isRecoverySupported() method. > Pros: Easy and clean > Cons: Possible backward compatibility issue due to aPI changes. (Is it true?) > 2. Call outputCommitter.setupTask(taskContext) from MRAM: The new > OutputCommitter will store the context in the class level variable and use it > from isRecoverySupported() > Props: No API changes. No backward compatibility issue. This call can be made > from MRAppMaster.getOutputCommitter() method for old API case. > Cons: Might not be very clean solution due to class level variable. > Please give your comments. -- This message was sent by Atlassian JIRA (v6.2#6252)