[
https://issues.apache.org/jira/browse/HBASE-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4365:
-------------------------
Attachment: 4365.txt
Here is a first cut.
It does not do the lookup of regions in a table across the cluster nor query zk
to find out how many nodes are in the mix. Its kinda hard to do this from a
RegionSplitPolicy context.
Instead, we count the number of regions that belong to a table on the current
regionserver. We then multiply the flushsize by this number and thats when
we'll split. If the multiplication produces a number > max filesize for a
region, we'll take maxfilesize.
If 1 region for a given table on a regionserver, we'll split on the first flush.
If 5 regions from same table on a regionserver, we'll split at 5 * 128M and so
on.
We could have the size grow more aggressively by squaring the count of regions;
that might make sense if cluster has lots of small tables -- in fact it might
be better altogether. What you all think?
If agreeable, I should make a new patch that makes this the default splitting
policy.
> Add a decent heuristic for region size
> --------------------------------------
>
> Key: HBASE-4365
> URL: https://issues.apache.org/jira/browse/HBASE-4365
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.94.0
> Reporter: Todd Lipcon
> Attachments: 4365.txt
>
>
> A few of us were brainstorming this morning about what the default region
> size should be. There were a few general points made:
> - in some ways it's better to be too-large than too-small, since you can
> always split a table further, but you can't merge regions currently
> - with HFile v2 and multithreaded compactions there are fewer reasons to
> avoid very-large regions (10GB+)
> - for small tables you may want a small region size just so you can
> distribute load better across a cluster
> - for big tables, multi-GB is probably best
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
