Here can be another problem - timestamp based keys or too many deletion.
Compaction removes most and regions become nearly empty.
lars hofhansl <mailto:[email protected]>
15 Aug 2014 22:12
HBase initially tries to spread the load out to more region servers by
splitting regions early when there aren't many (per table).
As more regions are seen split size approaches the configured max file
size.
This early splitting turned out to be a bit too aggressive (see
HBASE-10501, which modifies it a bit)... Even then 800 regions is a lot.
As others have asked, what's
hbase.hregion.max.filesize set to? Make sure it's at least 10gb.
-- Lars
________________________________
From: Juraj jiv <[email protected]>
To: [email protected]
Sent: Friday, August 15, 2014 2:08 AM
Subject: Re: Hbase region count and RS count for 2TB+
Hi,
size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data"
So i think yes, 650MB per RS. If its small then why hbase created so many
regions?
Thanks
JV
On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <[email protected]>
Juraj jiv <mailto:[email protected]>
15 Aug 2014 13:08
Hi,
size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data"
So i think yes, 650MB per RS. If its small then why hbase created so many
regions?
Thanks
JV
On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <[email protected]>
Mikhail Antonov <mailto:[email protected]>
15 Aug 2014 11:51
So you have 3200 regions in total, and that means each region is about
650mb in size only (if you have 2.2 Tb in total), unless I'm missing
something? If that's the case your regions are probably way too small.
-Mikhail
Juraj jiv <mailto:[email protected]>
15 Aug 2014 11:44
Hi all,
i read in documentation that max region count per region server should be
around 200. I unfortunately have currently 800 regions for RS.
Those tables are not presplitted (all handled by hbase), does manual
spliting helps in this case? hdfs fodler /hbase/data currently hold around
2.2TB of data and i have 4 region servers, is it enough or i need more RS?
Also will Hbase compact these tables automatically to have less regions?
Hbase version 0.96.1.1-cdh5.0.1
Thanks for any guidance.
JV
