Re: RS not processing any requests

Great this is good news! ~Jeff On 9/5/2012 6:49 PM, Ted Yu wrote: Backport has been done in HBASE-6724 Cheers On Wed, Sep 5, 2012 at 5:09 PM, Himanshu Vashishtha > wrote: It usually happens in a long running setup (at least for me). Can you throttle y

Re: RS not processing any requests

We can throttle our replication load. How do you do that? ;-) ~Jeff On 9/5/2012 6:09 PM, Himanshu Vashishtha wrote: It usually happens in a long running setup (at least for me). Can you throttle your load? Replication is evolving; I'd say update if you can (or backport the jiras?). Himanshu

Re: RS not processing any requests

Backport has been done in HBASE-6724 Cheers On Wed, Sep 5, 2012 at 5:09 PM, Himanshu Vashishtha wrote: > It usually happens in a long running setup (at least for me). Can you > throttle your load? > > Replication is evolving; I'd say update if you can (or backport the > jiras?). > > Himanshu >

Re: RS not processing any requests

It usually happens in a long running setup (at least for me). Can you throttle your load? Replication is evolving; I'd say update if you can (or backport the jiras?). Himanshu On Wed, Sep 5, 2012 at 5:53 PM, Jeff Whiting wrote: > hmm. So if we are on 0.92 what suggestion would you have to pre

Re: RS not processing any requests

hmm. So if we are on 0.92 what suggestion would you have to prevent the problem? ~Jeff On 9/5/2012 5:23 PM, Himanshu Vashishtha wrote: Number of PRI handlers are governed by "hbase.regionserver.metahandler.count"; default is 10. Increasing their number will not solve it, but will delay its o

Re: RS not processing any requests

Number of PRI handlers are governed by "hbase.regionserver.metahandler.count"; default is 10. Increasing their number will not solve it, but will delay its occurring (i don't know about your load etc). Another related jira is HBase-6550. Some more context for your use case: http://search-hadoop.

Re: RS not processing any requests

It looks like that is problem we are having. We are on 0.92 so we don't get the patch. But one solution seems to be increasing the privileged handlers. How do we increase the number of privilege handlers? ~Jeff On 9/5/2012 4:47 PM, Himanshu Vashishtha wrote: Your RS priority handlers are b

Re: RS not processing any requests

Yes we are running hbase replication. ~Jeff On 9/5/2012 4:47 PM, Himanshu Vashishtha wrote: Your RS priority handlers are blocked on meta lookup, so it becomes unresponsive. Looks like you hitting https://issues.apache.org/jira/browse/HBASE-6165 You running HBase replication? just confirming.

Re: RS not processing any requests

I work with Nathaniel and can answer those questions. We are using Sun's jvm. $ java -version java version "1.6.0_21" Java(TM) SE Runtime Environment (build 1.6.0_21-b06) Java HotSpot(TM) 64-Bit Server VM (build 17.0-b16, mixed mode) We also tried one node on a newer version but saw the same th

Re: RS not processing any requests

Your RS priority handlers are blocked on meta lookup, so it becomes unresponsive. Looks like you hitting https://issues.apache.org/jira/browse/HBASE-6165 You running HBase replication? just confirming. Himanshu On Wed, Sep 5, 2012 at 4:39 PM, Stack wrote: > On Wed, Sep 5, 2012 at 2:58 PM, Nathan

Re: RS not processing any requests

On Wed, Sep 5, 2012 at 2:58 PM, Nathaniel Cook wrote: > We ran a jstack on the both the RS process and the hbase shell process > trying to do the scan. > > Jstack log for RS: > http://pastebin.com/9Y9t5ERE > What JVM (I don't know what (20.10-b01 mixed mode) is). I see a bunch of this: "PRI IP

RS not processing any requests

We are experiencing a problem where RS are locking up and not processing any requests. Restarting the RS will fix the problem and operations will continue as normal. We are experiencing this issue under load and on two different clusters. We are importing existing data via the hbase mapreduce impor