Kevin, On Fri, Oct 16, 2009 at 9:50 AM, Tatsuya Kawano wrote: > Maybe you can workaround this by adding more RPC handlers (increase > the value of "hbase.regionserver.handler.count" in hbase-site.xml) > I'm going to run the same test again with 70 RPC handlers, and let you > know the result.
I ran the test with the proposed workaround (100 RPC handlers as Andy suggested) and it worked fine. I was able to insert 40 million records without deadlock (by the way, I'm running 3 region servers.) You have 12 mappers running at the same time (4 tasktrackers x 3 map slots each), so having 100 RPC handlers on each indexed region server will be safe enough to avoid the deadlock. Andy, Thanks for adding the troubleshooting tip and filing the issue. Those tips are very helpful. Ryan, On Fri, Oct 16, 2009 at 9:55 AM, Ryan Rawson wrote: > In general for at-scale I am not sure I can recommend the indexed > features. You might get better miliage with hand maintaining the > indexes yourself. Thanks for your advice. I agree with you. Server managed secondary index is convenient, but making the client application to maintain secondary indices should scale better. Also, different applications will need different secondary indices, so it's more like application's responsibly to maintain these indices. Thanks, -- Tatsuya Kawano (Mr.) Tokyo, Japan
