Re: region size/count per regionserver

, 2011 3:57 PM Subject: Re: region size/count per regionserver Simple answer - 20 regions/server 2000 regions/cluster is a good rule of thumb if you can't profile your workload yet. You really want to ensure that 1) You need to limits the regions/cluster so the master can have

Re: region size/count per regionserver

: Tuesday, November 1, 2011 3:57 PM Subject: Re: region size/count per regionserver Simple answer - 20 regions/server 2000 regions/cluster is a good rule of thumb if you can't profile your workload yet. You really want to ensure that 1) You need to limits the regions/cluster so

Re: region size/count per regionserver

. -- Lars From: Nicolas Spiegelberg nspiegelb...@fb.com To: user@hbase.apache.org user@hbase.apache.org Cc: Karthik Ranganathan kranganat...@fb.com; Kannan Muthukkaruppan kan...@fb.com Sent: Tuesday, November 1, 2011 3:57 PM Subject: Re: region size/count per

Re: region size/count per regionserver

To: user@hbase.apache.org user@hbase.apache.org Cc: Karthik Ranganathan kranganat...@fb.com; Kannan Muthukkaruppan kan...@fb.com Sent: Tuesday, November 1, 2011 3:57 PM Subject: Re: region size/count per regionserver Simple answer - 20 regions/server 2000 regions/cluster is a good rule

region size/count per regionserver

HI all, My HBase cluster is 10 nodes, each node has 12core , 48G RAM, 24TB disk, 10GEthernet. My region size is 1GB. Any guidelines on how many regions can a RS handle comfortably? I vaguely remember reading some where to have no more than 1000 regions / server; that comes to 1TB / server.

Re: region size/count per regionserver

These days I think the recommendation is more like 20 regions per region server, and the region size set accordingly. The major caveat is that when you start compacting the bigger store files you can really take a massive IO hit, so most of the time major compactions are tuned to run only every

Re: region size/count per regionserver

optimizations for compactions in 0.92. In our case we have a pretty old setup and had way too many regions so we ran a few online merges to bring this down to like 80 regions/RS and it's working pretty well. J-D what is the region size you use? and is it 80 regions / table / region-server?

Re: region size/count per regionserver

On Tue, Nov 1, 2011 at 2:34 PM, Sujee Maniyam su...@sujee.net wrote: optimizations for compactions in 0.92. In our case we have a pretty old setup and had way too many regions so we ran a few online merges to bring this down to like 80 regions/RS and it's working pretty well. J-D what is

Re: region size/count per regionserver

On Tue, Nov 1, 2011 at 2:46 PM, Sujee Maniyam su...@sujee.net wrote: 20GB, compressed ?  If so is it LZO or Snappy? The region size is expressed in terms of size on disk, in our case it's LZOed. J-D

Re: region size/count per regionserver

Simple answer - 20 regions/server 2000 regions/cluster is a good rule of thumb if you can't profile your workload yet. You really want to ensure that 1) You need to limits the regions/cluster so the master can have a reasonable startup time can handle all the region state