thanks, Ryan,
I think I can enlarge the heap size a little, and meanwhile
shrink the write buffer sounds good too, but I am not sure we can disable
the block cache, since the table is already built, anyway I can do that?
I still have the question that if I add more regionservers, will
the load on current nodes be reduced to make the cluster run OK?
On Mon, Mar 1, 2010 at 4:08 PM, Ryan Rawson <ryano...@gmail.com> wrote:
> I would consider doing the following things:
>
> - disable the block cache - you dont have ram to spare. 4gb is really
> minimum (my laptop is 4gb ram and i decry how sucky I am occasionally)
> - run between 1000-2000m, with the cache disabled you are in better shape
> - consider adjusting the config values:
> <name>hbase.regionserver.global.memstore.upperLimit</name>
> <value>0.4</value>
>
> This implies that memstore (the write buffer) can occupy 40% of heap.
> This might be too aggressive with a small heap and CMS. Try changing
> that to 0.2 and:
>
> <name>hbase.regionserver.global.memstore.lowerLimit</name>
> <value>0.35</value>
>
> to 0.10 or so.
>
> Good luck!
> -ryan
>
> On Sun, Feb 28, 2010 at 11:59 PM, steven zhuang <zhuangxin8...@gmail.com>
> wrote:
> > hi, all,
> > We have some PCs(a dozens of them) with 4 cores CPU and 4GB ram
> > each, a hadoop instance is already running on these machines.
> >
> > Currently there is around 1.2GB ram left on every node.
> >
> > We want to setup a Hbase instance on these machines, with 3
> quorum
> > servers and 5 region servers, but we kept getting "out of memory" error
> when
> > the table grows big on some nodes.
> >
> > I know HBase is really a memory eater, my question is can we set
> > up a Hbase cluster on these PC if we add more region servers? how much
> ram
> > should we reserve to make hbase run happily?
> >
> > An option is we put hbase regionservers and Hadoop
> > datanode/tasktracker on different node respectively(Not tried yet), but I
> > don't know how much this would influence the performance, as we can not
> > benefit from the data locality.
> >
> > --
> > best wishes.
> > steven
> >
>
--
best wishes.
steven
