[
https://issues.apache.org/jira/browse/HBASE-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13905031#comment-13905031
]
Hudson commented on HBASE-10501:
--------------------------------
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #93 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/93/])
HBASE-10501 Improve IncreasingToUpperBoundRegionSplitPolicy to avoid too many
regions. (larsh: rev 1569507)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionSplitPolicy.java
> Improve IncreasingToUpperBoundRegionSplitPolicy to avoid too many regions
> -------------------------------------------------------------------------
>
> Key: HBASE-10501
> URL: https://issues.apache.org/jira/browse/HBASE-10501
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Priority: Critical
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.17
>
> Attachments: 10501-0.94-v2.txt, 10501-0.94-v3.txt, 10501-0.94-v4.txt,
> 10501-0.94-v5.txt, 10501-0.94.txt, 10501-trunk-v2.txt, 10501-trunk.txt
>
>
> During some (admittedly artificial) load testing we found a large amount
> split activity, which we tracked down the
> IncreasingToUpperBoundRegionSplitPolicy.
> The current logic is this (from the comments):
> "regions that are on this server that all are of the same table, squared,
> times the region flush size OR the maximum region split size, whichever is
> smaller"
> So with a flush size of 128mb and max file size of 20gb, we'd need 13 region
> of the same table on an RS to reach the max size.
> With 10gb file sized it is still 9 regions of the same table.
> Considering that the number of regions that an RS can carry is limited and
> there might be multiple tables, this should be more configurable.
> I think the squaring is smart and we do not need to change it.
> We could
> * Make the start size configurable and default it to the flush size
> * Add multiplier for the initial size, i.e. start with n * flushSize
> * Also change the default to start with 2*flush size
> Of course one can override the default split policy, but these seem like
> simple tweaks.
> Or we could instead set the goal of how many regions of the same table would
> need to be present in order to reach the max size. In that case we'd start
> with maxSize/goal^2. So if max size is 20gb and the goal is three we'd start
> with 20g/9 = 2.2g for the initial region size.
> [~stack], I'm especially interested in your opinion.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
