Hello all, thanks for replies. hbase.hregion.max.filesize is set to default size 1GB. I set it to 10GB as recommended.
JV On Fri, Aug 15, 2014 at 9:15 PM, Ted Yu <[email protected]> wrote: > In that case, HBASE-7403 would come to the rescue. > > Cheers > > > On Fri, Aug 15, 2014 at 12:09 PM, Andrey Stepachev <[email protected]> > wrote: > >> Here can be another problem - timestamp based keys or too many deletion. >> Compaction removes most and regions become nearly empty. >> >> lars hofhansl <[email protected]> >> 15 Aug 2014 22:12 >> HBase initially tries to spread the load out to more region servers by >> splitting regions early when there aren't many (per table). >> As more regions are seen split size approaches the configured max file >> size. >> This early splitting turned out to be a bit too aggressive (see >> HBASE-10501, which modifies it a bit)... Even then 800 regions is a lot. >> >> As others have asked, what's >> >> hbase.hregion.max.filesize set to? Make sure it's at least 10gb. >> >> >> -- Lars >> >> >> >> ________________________________ >> From: Juraj jiv <[email protected]> <[email protected]> >> To: [email protected] >> Sent: Friday, August 15, 2014 2:08 AM >> Subject: Re: Hbase region count and RS count for 2TB+ >> >> >> Hi, >> size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data" >> >> So i think yes, 650MB per RS. If its small then why hbase created so many >> regions? >> >> Thanks >> JV >> >> >> >> On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <[email protected]> >> <[email protected]> >> Juraj jiv <[email protected]> >> 15 Aug 2014 13:08 >> Hi, >> size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data" >> >> So i think yes, 650MB per RS. If its small then why hbase created so many >> regions? >> >> Thanks >> JV >> >> >> On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <[email protected]> >> <[email protected]> >> >> Mikhail Antonov <[email protected]> >> 15 Aug 2014 11:51 >> So you have 3200 regions in total, and that means each region is about >> 650mb in size only (if you have 2.2 Tb in total), unless I'm missing >> something? If that's the case your regions are probably way too small. >> >> -Mikhail >> >> >> >> >> >> Juraj jiv <[email protected]> >> 15 Aug 2014 11:44 >> Hi all, >> i read in documentation that max region count per region server should be >> around 200. I unfortunately have currently 800 regions for RS. >> Those tables are not presplitted (all handled by hbase), does manual >> spliting helps in this case? hdfs fodler /hbase/data currently hold around >> 2.2TB of data and i have 4 region servers, is it enough or i need more RS? >> >> Also will Hbase compact these tables automatically to have less regions? >> >> Hbase version 0.96.1.1-cdh5.0.1 >> Thanks for any guidance. >> >> JV >> >> >
