[ https://issues.apache.org/jira/browse/HBASE-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480587#comment-13480587 ]
Hadoop QA commented on HBASE-7018: ---------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12550120/HBASE-7018-trunk.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestHRegionInfo Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3098//console This message is automatically generated. > Fix and Improve TableDescriptor caching for bulk assignment > ----------------------------------------------------------- > > Key: HBASE-7018 > URL: https://issues.apache.org/jira/browse/HBASE-7018 > Project: HBase > Issue Type: Bug > Components: regionserver > Reporter: Gregory Chanan > Assignee: Gregory Chanan > Fix For: 0.94.3, 0.96.0 > > Attachments: HBASE-7018-94.patch, HBASE-7018-94-v2.patch, > HBASE-7018-trunk.patch > > > HBASE-6214 backported HBASE-5998 (Bulk assignment: regionserver optimization > by using a temporary cache for table descriptors when receiving an open > regions request), but it's buggy on 0.94 (0.96 appears correct): > {code} > HTableDescriptor htd = null; > if (htds == null) { > htd = this.tableDescriptors.get(region.getTableName()); > } else { > htd = htds.get(region.getTableNameAsString()); > if (htd == null) { > htd = this.tableDescriptors.get(region.getTableName()); > htds.put(region.getRegionNameAsString(), htd); > } > } > {code} > i.e. we get the tableName from the map but write the regionName. > Even fixing this, it looks like there are areas for improvement: > 1) FSTableDescriptors already has a cache (though it goes to the NameNode > each time through to check we have the latest copy. May as well combine > these two caches, might be a performance win as well since we don't need to > write to multiple caches. > 2) FSTableDescriptors makes two RPCs to the NameNode when it encounters a new > table. So the total number of RPCs necessary for a bulk assign (without > caching is): > #regions + #tables > (with caching): > min(#regions,#tables) + #tables = #tables + #tables = 2 * #tables > We can make this only one RPC, yielding: > #tables > Probably not a big deal for most users, but in a multi-tenant situation where > the number of regions being bulk assigned approaches the number of tables > being bulk assigned, this could be a nice performance win. > Benchmarks coming. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira