Re: Very high read rate in a single RegionServer

2015-01-28 Thread Stack
On Tue, Jan 27, 2015 at 12:31 PM, Pun Intended punintende...@gmail.com wrote: On Tue, Jan 27, 2015 at 12:58 PM, Stack st...@duboce.net wrote: On Tue, Jan 27, 2015 at 9:34 AM, Pun Intended punintende...@gmail.com wrote: The majority of the MR jobs are doing heavy reads/scans. Also doing

Re: After compression, the table folders under hdfs://hbase are empty

2015-01-28 Thread Ted Yu
I am not familiar with CDH deployment. Log onto the server where regionserver process is running and issue: ps aux | grep regionserver In the output look for '-Dhbase.log.dir=' which would tell you the location of the log. Cheers On Wed, Jan 28, 2015 at 7:37 AM, kennyut ken.oy...@gmail.com

Tool to to execute an benchmark for HBase.

2015-01-28 Thread Guillermo Ortiz
Hi, I'd like to do some benchmarks fo HBase but I don't know what tool could use. I started to make some code but I guess that there're some easier. I've taken a look to JMeter, but I guess that I'd attack directly from Java, JMeter looks great but I don't know if it fits well in this scenario.

Re: Tool to to execute an benchmark for HBase.

2015-01-28 Thread Ted Yu
Guillermo: If you use hbase 0.98.x, please consider Andrew's ycsb repo: https://github.com/apurtell/ycsb/tree/new_hbase_client Cheers On Wed, Jan 28, 2015 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: You can use ycsb for this purpose.See here

Re: Tool to to execute an benchmark for HBase.

2015-01-28 Thread Nishanth S
You can use ycsb for this purpose.See here https://github.com/brianfrankcooper/YCSB/wiki/Getting-Started -Nishanth On Wed, Jan 28, 2015 at 1:37 PM, Guillermo Ortiz konstt2...@gmail.com wrote: Hi, I'd like to do some benchmarks fo HBase but I don't know what tool could use. I started to make

Re: Tool to to execute an benchmark for HBase.

2015-01-28 Thread Guillermo Ortiz
I was checking that web, do you know if there's another possibility since last updated for Cassandra was two years ago and I'd like to compare bothof them with kind of same tool/code. 2015-01-28 22:10 GMT+01:00 Ted Yu yuzhih...@gmail.com: Guillermo: If you use hbase 0.98.x, please consider

java API bug ? --- client.HTable init unnecessarily tries hadoop

2015-01-28 Thread Yang
we have a standalone java code, which simply tries to insert one record into an existing hbase table. here it got the following error:but it is able to proceed. so this means the following operation which triggered the error is useless ??? if so, shouldn't the useless code be removed? thanks

Re: java API bug ? --- client.HTable init unnecessarily tries hadoop

2015-01-28 Thread Ted Yu
Is namenode running on my-hbase-master.com http://my-hbase-master.com:8020/apps/hbase/data/lib ? Which hbase release are you using ? I assume hbase-site.xml is in the classpath of your app. What's the value for hbase.rootdir ? Cheers On Wed, Jan 28, 2015 at 4:10 PM, Yang tedd...@gmail.com

Re: Does 'online region merge' make regions unavailable for some time?

2015-01-28 Thread Otis Gospodnetic
Thanks everyone for the feedback. What we ended up implementing is something that does these merges programmatically, but limits merges to regions that live on the same RS. They take on the order of just a few seconds for us. Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log

Re: Tool to to execute an benchmark for HBase.

2015-01-28 Thread Ted Yu
Maybe ask on Cassandra mailing list for the benchmark tool they use ? Cheers On Wed, Jan 28, 2015 at 1:23 PM, Guillermo Ortiz konstt2...@gmail.com wrote: I was checking that web, do you know if there's another possibility since last updated for Cassandra was two years ago and I'd like to

Re: After compression, the table folders under hdfs://hbase are empty

2015-01-28 Thread kennyut
I can find the hbase version from hbase shell (please see below): hbase(main):001:0 version 0.98.1-cdh5.1.3, rUnknown, Tue Sep 16 20:19:34 PDT 2014 But where I can find my HBase region server log? I used all default option when installing HBase, including all configurations. Thanks for your

Re: about HBase versions

2015-01-28 Thread sirajudeen.v
Is the HBase version is reliable solution to use in critical data. In case I am not going to store duplicate data (same row key and time stamp). -- View this message in context: http://apache-hbase.679495.n3.nabble.com/about-HBase-versions-tp4024747p4067937.html Sent from the HBase User

Re: After compression, the table folders under hdfs://hbase are empty

2015-01-28 Thread kennyut
In the /var/log/hbase folder, I can find two files: 1. hbase-cmf-hbase-HBASERESTSERVER-ip-172-x-x-x.ec2.internal.log.out 2. hbase-cmf-hbase-MASTER-ip-172-x-x-x.ec2.internal.log.out The '1' file was updated several days before, so it won't help. The '2' file only records the hbase log when I use

Re: about HBase versions

2015-01-28 Thread kennyut
I am also working with a very similar project. I tried two version for time-series data: 1. use id-timestamp as rowkey: It's good for loading new inputs and also good for search. But bad for the storage and slow when doing aggregation. 2. use id as rowkey and use array string for time and

Re: about HBase versions

2015-01-28 Thread Ted Yu
Can you provide a bit more detail on your use case ? If row key and timestamp are different across writes, there would be 1 version. Cheers On Wed, Jan 28, 2015 at 7:25 AM, sirajudeen.v sirajudee...@tcs.com wrote: Is the HBase version is reliable solution to use in critical data. In case I

Re: After compression, the table folders under hdfs://hbase are empty

2015-01-28 Thread Jean-Marc Spaggiari
Have you looked a the default /var/log directory to see what you have there? JM 2015-01-28 9:11 GMT-08:00 Ted Yu yuzhih...@gmail.com: I am not familiar with CDH deployment. Log onto the server where regionserver process is running and issue: ps aux | grep regionserver In the output look