Digging into this further, I see the following in HTablePool:
public void putTable(HTableInterface table) {
LinkedList<HTableInterface> queue =
tables.get(Bytes.toString(table.getTableName()));
synchronized(queue) {
if(queue.size() >= maxSize) {
// release table instance since we're not reusing it
this.tableFactory.releaseHTableInterface(table);
return;
}
queue.add(table);
}
}
maxSize is hard coded to 10, so I think if there are more than 10 simultaneous
requests, future requests will get brand-new tables, lacking any cache.
Does this sound plausible? I can submit a JIRA asking for maxSize to be a
config param if so. We have >> 10 simultaneous requests.
----- Original Message -----
From: Ben West <bwsithspaw...@yahoo.com>
To: Stack <st...@duboce.net>; "user@hbase.apache.org" <user@hbase.apache.org>
Cc:
Sent: Friday, December 9, 2011 10:52 AM
Subject: Re: Meta region hotspotting
Thanks Stack.
Is there any way I can log requests going to .META. vs. region servers? I've
tried setting various portions of HBase to DEBUG but I haven't found the magic
combo yet.
----- Original Message -----
From: Stack <st...@duboce.net>
To: user@hbase.apache.org; Ben West <bwsithspaw...@yahoo.com>
Cc:
Sent: Thursday, December 8, 2011 6:24 PM
Subject: Re: Meta region hotspotting
On Thu, Dec 8, 2011 at 4:05 PM, Ben West <bwsithspaw...@yahoo.com> wrote:
> We have a cluster with four region servers and about 2,000 regions. We're
> using the REST server, and we've noticed that whatever region is hosting META
> gets 3-5x the number of requests that the other regions do.
>
> It's my understanding that the client should cache the row start/end
> locations, and hence should need to go to META only rarely after a little
> bit. But I'm sustaining tens of thousands of requests per second for many
> minutes and they all still seem to hit META - does anyone know why this would
> be?
>
Your understanding is corrrect. I'd think that the caching would
eventually pick up all 2k region locations unless REST is making a
new client on every invocation, or every few invocations (I've not
checked).
St.Ack
