Re: increasing hbase get latencies

2011-06-13 Thread Abhijit Pol
Thanks Stack. We will try those patches and upgrade to 0.90.3 and see how things improve. I will update in few days. GC pause don't follow any increasing pattern, so we can eliminate that. Store files, I had confusing input. We have 300 regions, 2 column families, and _total_ 1300-1500 files. Thi

Re: increasing hbase get latencies

2011-06-10 Thread Stack
On Fri, Jun 10, 2011 at 10:10 AM, Abhijit Pol wrote: >> performed table flush at around peak timeouts 35%. Timeouts went up a bit > during flush and then dropped to 28% mark (compared to 10% timeouts after > restart of server). They started climbing up again and reached 35% mark in > an hour or so

Re: increasing hbase get latencies

2011-06-10 Thread Stack
ein (via > Tom White) > > > --- On Wed, 6/8/11, Stack wrote: > >> From: Stack >> Subject: Re: increasing hbase get latencies >> To: user@hbase.apache.org >> Date: Wednesday, June 8, 2011, 11:31 PM >> On Wed, Jun 8, 2011 at 4:39 PM, >> Abhijit Po

Re: increasing hbase get latencies

2011-06-10 Thread Abhijit Pol
e their worth by hitting back. - Piet Hein > (via Tom White) > > > --- On Wed, 6/8/11, Stack wrote: > > > From: Stack > > Subject: Re: increasing hbase get latencies > > To: user@hbase.apache.org > > Date: Wednesday, June 8, 2011, 11:31 PM > > On Wed, Jun 8

Re: increasing hbase get latencies

2011-06-10 Thread Abhijit Pol
Thanks for you reply, stack. I have answers inline. By timeout % we refer to, % of requests that exceeds our acceptable latency threshold. On Wed, Jun 8, 2011 at 11:31 PM, Stack wrote: > What happens if you flush that table/region when its slow (you can do > it from the shell). Does the latency

Re: increasing hbase get latencies

2011-06-10 Thread Andrew Purtell
, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) --- On Wed, 6/8/11, Stack wrote: > From: Stack > Subject: Re: increasing hbase get latencies > To: user@hbase.apache.org > Date: Wednesday, June 8, 2011, 11:31 PM > On Wed, J

Re: increasing hbase get latencies

2011-06-08 Thread Stack
On Wed, Jun 8, 2011 at 4:39 PM, Abhijit Pol wrote: > Recently we observed that our "get" latencies keep increasing over the > period (and eventually flatten out at higher value) and if we restart hbase > server, latencies go back to good state (low values) and start increasing > again. > What hap

increasing hbase get latencies

2011-06-08 Thread Abhijit Pol
We are on hbase 0.90 and using hbase for a while to perform high volume data lookup using hbase client (no map-reduce involved). Recently we observed that our "get" latencies keep increasing over the period (and eventually flatten out at higher value) and if we restart hbase server, latencies go b