thanks, J.D.
I am still not sure about the second question, from the log I
can see lines like:
*org.apache.hadoop.hbase.regionserver.Store: loaded
/user/ccenterq/hbase/XXX/1702600912/queries/1289015788537930719,
isReference=false, sequence id=1389720128, length=**175533391**,
majorCompaction=true (this is the region data, not the index, right?)*
I do have some region really big, with millions of columns in
one column family, but isn't this length a little too big.
About the third one, I am actually not very clear of how memory is
used in Hbase, if it's only the few KBs by holding region info, it won't
release right?
Since we adding more regionserver into our cluster, the chance of
OOME is much less, but still there are some(we have only 1GB heap for the
regoinserver), I wonder which part of Hbase eats most of the heap
memory? The region data takes most, right?
I think there should be some rule to cache/write-back the region
data, but now it seems that the Hbase doesn't detect the Memory usage status
thus cannot release the memory in time.
On Fri, Mar 5, 2010 at 10:00 AM, Jean-Daniel Cryans <[email protected]>wrote:
> Inline.
>
> J-D
>
> On Thu, Mar 4, 2010 at 5:53 PM, steven zhuang
> <[email protected]> wrote:
> > greetings,
> >
> > I have a cluster which contains a dozens of PCs with 4cores
> and
> > 4GB ram each, at first there were only 5 regionservers in the cluster,
> now
> > we have added 18 regionservers into the cluseter.
> > But after reading the log, I have some questions, hope you
> guys
> > can help me out.
> >
> > 1. the cluster will reassign the regions around old
> > regionservers and newly added regionservers because of auto-balancing,
> > right?
>
> Yep. It's a bit intense and we plan on making it less disruptive.
>
> >
> > 2. I saw a lot of regions OPEN messages, does this means the
> > regionserver will load the whole regoin into memory or just portion of
> it,
> > or just the index(?I don't know what is in the index data)?
>
> Just the region information and files indexes.
>
> >
> > 3. will the memory be released after the loaded region opened
> is
> > not needed anymore, Coz I don't see anything from the log.
>
> Which memory? The few KBs used to store the region's information? Or
> are you talking about the memory taken by regions that moved to other
> region servers? In this case it's really just the java garbage
> collector that takes care of it.
>
