Pedro Gandola <pedro.gandola@...> writes:
>
> Hi Ted,
>
> Thanks,
> I think I got the problem, I'm using *IncreasingToUpperBoundRegionSplitPolicy
> (default)* instead *ConstantSizeRegionSplitPolicy* which in my use case is
> what I want.
>
> Cheers
> Pedro
>
> On Mon, Feb 15, 2016 at 5:22 PM, Ted Yu <yuzhihong@...> wrote:
>
> > Can you pastebin region server log snippet around the time when the split
> > happened ?
> >
> > Was the split on data table or index table ?
> >
> > Thanks
> >
> > > On Feb 15, 2016, at 10:22 AM, Pedro Gandola <pedro.gandola@...>
> > wrote:
> > >
> > > Hi,
> > >
> > > I have a cluster using *HBase 1.1.2* where I have a table and a local
> > index
> > > (using *Apache Phoenix 4.6*) in total both tables have *300 regions*
> > > (aprox: *18 regions per server*), my* hbase.hregion.max.filesize=30GB
> > *and
> > > my region sizes are now *~4.5GB compressed (~7GB uncompressed)*. However
> > > each time I restart a RS sometimes a region gets split. This is
> > unexpected
> > > because my key space is uniform (using MD5) and if the problem was my
> > > *region.size
> > >> * *hbase.hregion.max.filesize *I would expect to have all the regions or
> > > almost all splitting but this only happens when I restart a RS and it
> > > happens only for 1 or 2 regions.
> > >
> > > What are the different scenarios where a region can split?
> > >
> > > What are the right steps to restart a region server in order to avoid
> > these
> > > unexpected splits?
> > >
> > > Thank you,
> > > Cheers
> > > Pedro
> >
>
Thanks Pedro for giving your solution.
i see the same issue during Hbase restarts. unexpected region splits.
i believe it is because the *IncreasingToUpperBoundRegionSplitPolicy* is basing
his calculation on the amount of ONLINE regions.
but while the RS is starting only a couple of regions are online YET.
so the policy things it would be no problem to add another region
since 'there are only a few'.
(while there are actually already are 330 for that RS for that phoenix table...
yes i know i need to merge regions.
but this problem got out of hand unnoticed for some time now here)
could HBase block split region decision until it is fully up and running?
Hbase 1.0.0 logs. (check mainly the last line)
Mar 24, 11:06:41.494 AM INFO
org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher
Flushed, sequenceid=69436099, memsize=303.3 K, hasBloomFilter=true, into tmp
file
hdfs://ns/hbase/data/default/CUSTOMER/60af2857a7980ce4f1ac602dd83e05a6/.tmp/
0fd4988f24f24d5d9887c542182efccc
Mar 24, 11:06:41.529 AM INFO
org.apache.hadoop.hbase.regionserver.HStore
Added hdfs://-ns/hbase/data/default/CUSTOMER/
ff4ecd56e6b06f228404f05f171f8282/0/1d05cf9cac4c46008e47e3578e7a18d6,
entries=235, sequenceid=22828972, filesize=5.5 K
Mar 24, 11:06:41.561 AM INFO
org.apache.hadoop.hbase.regionserver.HStore
Completed compaction of 3 (all) file(s) in s of CUSTOMER,\x0A0+\xF6\
xD8,1457121856469.183f6134683e0213ccb15558a56f7c02.
into 730489295b8c42afaec4a3b8bc38c915(size=1.4 M),
total size for store is 1.4 M. This selection was in queue for
0sec, and took 0sec to execute.
Mar 24, 11:06:41.561 AM INFO
org.apache.hadoop.hbase.regionserver.CompactSplitThread
Completed compaction: Request = regionName=CUSTOMER,
\x0A0+\xF6\xD8,1457121856469.183f6134683e0213ccb15558a56f7c02.,
storeName=s, fileCount=3, fileSize=1.7 M, priority=7, time=1456532583179472;
duration=0sec
Mar 24, 11:06:41.562 AM DEBUG
org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy
ShouldSplit because IB size=3269370636, sizeToCheck=2147483648,
regionsWithCommonTable=2
i will also revert back to the ConstantSizeRegionSplitPolicy
Regards,
