>From IncreasingToUpperBoundRegionSplitPolicy#configureForRegion():
initialSize = conf.getLong("hbase.increasing.policy.initial.size", -1);
...
if (initialSize <= 0) {
initialSize = 2 * conf.getLong(HConstants.HREGION_MEMSTORE_FLUSH_SIZE,
HTableDescriptor.
DEFAULT_MEMSTORE_FLUSH_SIZE);
If you haven't changed the value for
"hbase.increasing.policy.initial.size", the last two lines should have been
executed.
initialSize would be 2GB in that case according to the config you listed.
FYI
On Fri, Aug 26, 2016 at 3:23 PM, yeshwanth kumar <[email protected]>
wrote:
> Hi we are using CDH 5.7 HBase 1.2
>
> we are doing a performance testing over HBase through regular Load, which
> has 4 Region Servers.
>
> Input Data is compressed binary files around 2TB, which we process and
> write as Key-Value pairs to HBase.
> the output data size in HBase is almost 4 times around 8TB, because we are
> writing as text.
> this process is a Map-Reduce Job,
>
> when we are doing the load, we observed there's a lot of GC happening on
> Region Server's ,so we changed couple of parameters to decrease the GC
> time.
>
> we increased the flush size to 128MB to 1 GB and compactionThreshold to 50
> and regionserver.maxlogs to 42
> following are the configuration we changed from default.
>
>
> hbase.hregion.memstore.flush.size = 1 GB
> hbase.hstore.max.filesize=10GB
> hbase.hregion.preclose.flush.size= 50 MB
>
> hbase.hstore.compactionThreshold=50
> hbase.regionserver.maxlogs=42
>
> after the load, we observed that HBase table has only 4 regions with each
> of size around 2.5 TB
>
> i am trying to understand, what configuration parameter caused this issue.
>
> i was going through this article
> http://hortonworks.com/blog/apache-hbase-region-splitting-and-merging/
>
> Region split policy in our HBase is
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSp
> litPolicy
> according to Region Split policy, Region Server should create regions when
> the region size limit is exceeded.
> can some one explain me the root cause.
>
>
> Thanks,
> Yeshwanth
>
