[ https://issues.apache.org/jira/browse/MAPREDUCE-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354337#comment-14354337 ]
Hadoop QA commented on MAPREDUCE-4961: -------------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12572711/MAPREDUCE-4961.patch against trunk revision 47f7f18. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5267//console This message is automatically generated. > Map reduce running local should also go through ShuffleConsumerPlugin for > enabling different MergeManager implementations > ------------------------------------------------------------------------------------------------------------------------- > > Key: MAPREDUCE-4961 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4961 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Jerry Chen > Assignee: Jerry Chen > Attachments: MAPREDUCE-4961.patch, MAPREDUCE-4961.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > MAPREDUCE-4049 provide the ability for pluggable Shuffle and MAPREDUCE-4080 > extends Shuffle to be able to provide different MergeManager implementations. > While using these pluggable features, I find that when a map reduce is > running locally, a RawKeyValueIterator was returned directly from a static > call of Merge.merge, which break the assumption that the Shuffle may provide > different merge methods although there is no copy phase for this situation. > The use case is when I am implementating a hash-based MergeManager, we don't > need sort in map side, while when running the map reduce locally, the > hash-based MergeManager will have no chance to be used as it goes directly to > Merger.merge. This makes the pluggable Shuffle and MergeManager incomplete. > So we need to move the code calling Merger.merge from Reduce Task to > ShuffleConsumerPlugin implementation, so that the Suffle implementation can > decide how to do the merge and return corresponding iterator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)