[ https://issues.apache.org/jira/browse/HBASE-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209708#comment-14209708 ]
Hadoop QA commented on HBASE-12394: ----------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12681309/HBASE-12394-v5.patch against trunk revision . ATTACHMENT ID: 12681309 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 6 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1 warning messages. {color:green}+1 checkstyle{color}. The applied patch does not increase the total number of checkstyle errors {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestRegionReplicas {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): at org.apache.hadoop.hbase.regionserver.TestRegionReplicas.testVerifySecondaryAbilityToReadWithOnFiles(TestRegionReplicas.java:421) at org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:183) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/11661//console This message is automatically generated. > Support multiple regions as input to each mapper in map/reduce jobs > ------------------------------------------------------------------- > > Key: HBASE-12394 > URL: https://issues.apache.org/jira/browse/HBASE-12394 > Project: HBase > Issue Type: Improvement > Components: mapreduce > Affects Versions: 2.0.0, 0.98.6.1 > Reporter: Weichen Ye > Attachments: HBASE-12394-v2.patch, HBASE-12394-v3.patch, > HBASE-12394-v4.patch, HBASE-12394-v5.patch, HBASE-12394.patch, HBase-12394 > Document.pdf > > > Welcome to the ReviewBoard :https://reviews.apache.org/r/27519/ > The Latest Patch is "Diff Revision 2 (Latest)" > For Hadoop cluster, a job with large HBase table as input always consumes a > large amount of computing resources. For example, we need to create a job > with 1000 mappers to scan a table with 1000 regions. This patch is to support > one mapper using multiple regions as input. > In order to support multiple regions for one mapper, we need a new property > in configuration--"hbase.mapreduce.scan.regionspermapper" > hbase.mapreduce.scan.regionspermapper controls how many regions used as input > for one mapper. For example,if we have an HBase table with 300 regions, and > we set hbase.mapreduce.scan.regionspermapper = 3. Then we run a job to scan > the table, the job will use only 300/3=100 mappers. > In this way, we can control the number of mappers using the following formula. > Number of Mappers = (Total region numbers) / > hbase.mapreduce.scan.regionspermapper > This is an example of the configuration. > <property> > <name>hbase.mapreduce.scan.regionspermapper</name> > <value>3</value> > </property> > This is an example for Java code: > TableMapReduceUtil.initTableMapperJob(tablename, scan, Map.class, Text.class, > Text.class, job); > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)