[ https://issues.apache.org/jira/browse/HBASE-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012252#comment-17012252 ]
Michael Stack commented on HBASE-21065: --------------------------------------- Put up a patch. Lets see how it does. > Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters on meta while we > are at it) > ------------------------------------------------------------------------------------- > > Key: HBASE-21065 > URL: https://issues.apache.org/jira/browse/HBASE-21065 > Project: HBase > Issue Type: Improvement > Components: meta, Performance > Reporter: Michael Stack > Assignee: Michael Stack > Priority: Major > > Some users end up hitting meta hard. Bulk is probably because our client goes > to meta too often, and the real 'fix' for a saturated meta is splitting it, > but the encoding that came in with HBASE-16213, ROW_INDEX_V1, could help in > the near term. It adds an index on hfile blocks and helped improve random > reads against user-space tables (less compares as we used index to go direct > to requested Cells rather than look at each Cell in turn until we found what > we wanted -- see RN on HBASE-16213 for citation). > I also noticed code-reading that we don't enable blooms on hbase:meta tables; > that could save some CPU and speed things up a bit too: > {code} > // Disable blooms for meta. Needs work. Seems to mess w/ > getClosestOrBefore. > .setBloomFilterType(BloomType.NONE) > {code} > This issue is about doing a bit of perf compare of encoding *on* vs current > default (and will check diff in size of indexed blocks). > Meta access is mostly random-read I believe (A review of a user's access > showed this so at least for their workload). The nice addition, HBASE-19722 > Meta query statistics metrics source, would help verify if it saw some usage > on a prod cluster. > If all is good, I'd like to make a small patch, one that could be easily > backported, with minimal changes in it. > As is, its all a little awkward as the meta table schema is hard-coded and > meta is immutable -- stuff we'll have to fix if we want to split meta -- so > in the meantime it requires a code change to enable (and a backport of > HBASE-16213 -- this patch is in 1.4.0 only currently, perhaps that is > enough). Code change to enable is small: > {code} > diff --git > a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java > > b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java > index 28c7ec3c2f..8f08f94dc1 100644 > --- > a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java > +++ > b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java > @@ -160,6 +160,7 @@ public class FSTableDescriptors implements > TableDescriptors { > .setScope(HConstants.REPLICATION_SCOPE_LOCAL) > // Disable blooms for meta. Needs work. Seems to mess w/ > getClosestOrBefore. > .setBloomFilterType(BloomType.NONE) > + > .setDataBlockEncoding(org.apache.hadoop.hbase.io.encoding.DataBlockEncoding.ROW_INDEX_V1) > .build()) > > .setColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(HConstants.TABLE_FAMILY) > .setMaxVersions(conf.getInt(HConstants.HBASE_META_VERSIONS, > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)