Re: Hbase scalability performance

2012-12-23 Thread Mohammad Tariq
Hello Dalia, You can go the Hbase webUI to see the details, as Ted has specified earlier. But if you really want to monitor everything properly I would suggest to configure Ganglia to capture the metrics. To do a quick check you can also use "status" command from the Hbase shell. hbase> stat

Re: Hbase scalability performance

2012-12-23 Thread Dimitry Goldin
Hi, On 23.12.2012 14:38, Dalia Sobhy wrote: So do you have an example of multithreading program, because I am using the read-made Java API not thrift server, so I don't know how to write a multithreaded program using this API. You should take a loot at YCSB (https://github.com/brianfrankco

RE: Hbase scalability performance

2012-12-23 Thread Dalia Sobhy
I am using 3 region servers. Hbase version: 0.92 Cloudera Manager: 4.1 How to know the load is balanced Ted? > Date: Sat, 22 Dec 2012 08:06:59 -0800 > Subject: Re: Hbase scalability performance > From: yuzhih...@gmail.com > To: user@hbase.apache.org > > By '3 datanodes&

RE: Hbase scalability performance

2012-12-23 Thread Dalia Sobhy
(not virtual ones), and greater bandwidth than 100Mbps. They told me those two issues caused this performance. But upon trial, I found the same case. > From: donta...@gmail.com > Date: Sat, 22 Dec 2012 23:09:54 +0530 > Subject: Re: Hbase scalability performance > To: user@hbase.

RE: Hbase scalability performance

2012-12-23 Thread Dalia Sobhy
So do you have an example of multithreading program, because I am using the read-made Java API not thrift server, so I don't know how to write a multithreaded program using this API. > Date: Sat, 22 Dec 2012 08:50:56 -0800 > Subject: Re: Hbase scalability performance

Re: Hbase scalability performance

2012-12-22 Thread Mohammad Tariq
I totally agree with Michael. I was about to point out the same thing. Probability of RS hotspotting is high when we have sequential keys. Even if everything is balanced and your cluster is very well configured you might end up with this issue. Best Regards, Tariq +91-9741563634 https://mtariq.jux

Re: Hbase scalability performance

2012-12-22 Thread Mohit Anchlia
Also, check how balanced your region servers are accross all the nodes On Sat, Dec 22, 2012 at 8:50 AM, Varun Sharma wrote: > Note that adding nodes will improve throughput and not latency. So, if your > client application for benchmarking is single threaded, do not expect an > improvement in nu

Re: Hbase scalability performance

2012-12-22 Thread Varun Sharma
Note that adding nodes will improve throughput and not latency. So, if your client application for benchmarking is single threaded, do not expect an improvement in number of reads per second by just adding nodes. On Sat, Dec 22, 2012 at 8:23 AM, Michael Segel wrote: > I thought it was Doug Miel w

Re: Hbase scalability performance

2012-12-22 Thread Michael Segel
I thought it was Doug Miel who said that HBase doesn't start to shine until you had at least 5 nodes. (Apologies if I misspelled Doug's name.) I happen to concur and if you want to start testing scalability, you will want to build a bigger test rig. Just saying! Oh and you're going to have

Re: Hbase scalability performance

2012-12-22 Thread Ted Yu
By '3 datanodes', did you mean that you also increased the number of region servers to 3 ? When your test was running, did you look at Web UI to see whether load was balanced ? You can also use Ganglia for such purpose. What version of HBase are you using ? Thanks On Sat, Dec 22, 2012 at 7:43 A

Hbase scalability performance

2012-12-22 Thread Dalia Sobhy
Dear all, I am testing a simple hbase application on a cluster of multiple nodes. I am especially testing the scalability performance, by measuring the time taken for random reads Data size: 200,000 row Row key : 0,1,2 very simple row key incremental But i don't know why by increasing the clus