[ https://issues.apache.org/jira/browse/MAPREDUCE-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563385#comment-13563385 ]
Hadoop QA commented on MAPREDUCE-4961: -------------------------------------- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566612/MAPREDUCE-4961.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3279//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3279//console This message is automatically generated. > Map reduce running local should also go through ShuffleConsumerPlugin for > enabling different MergeManager implementations > ------------------------------------------------------------------------------------------------------------------------- > > Key: MAPREDUCE-4961 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4961 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: trunk > Reporter: Jerry Chen > Assignee: Jerry Chen > Attachments: MAPREDUCE-4961.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > MAPREDUCE-4049 provide the ability for pluggable Shuffle and MAPREDUCE-4080 > extends Shuffle to be able to provide different MergeManager implementations. > While using these pluggable features, I find that when a map reduce is > running locally, a RawKeyValueIterator was returned directly from a static > call of Merge.merge, which break the assumption that the Shuffle may provide > different merge methods although there is no copy phase for this situation. > The use case is when I am implementating a hash-based MergeManager, we don't > need sort in map side, while when running the map reduce locally, the > hash-based MergeManager will have no chance to be used as it goes directly to > Merger.merge. This makes the pluggable Shuffle and MergeManager incomplete. > So we need to move the code calling Merger.merge from Reduce Task to > ShuffleConsumerPlugin implementation, so that the Suffle implementation can > decide how to do the merge and return corresponding iterator. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira