You can read metrics [0] from JMX directly [1] or use Ganglia [2] or other third-party tools like [3] (I'm a little biased here;)).
[0] http://hbase.apache.org/book.html#hbase_metrics [1] http://hbase.apache.org/metrics.html [2] http://wiki.apache.org/hadoop/GangliaMetrics [3] http://sematext.com/spm/hbase-performance-monitoring/index.html Note, that metrics values may seem a bit ugly/weird: as they say, you have to refer to Lars' book HBase in Action to understand how some of them calculated. There's an ongoing work towards revising metrics, they should look much better in next releases. Alex Baranau ------ Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch - Solr On Fri, Jul 27, 2012 at 2:21 PM, syed kather <in.ab...@gmail.com> wrote: > Thank you so much for your valuable information. I had not yet used any > monitoring tool .. can please suggest me a good monitor tool . > > Syed Abdul kather > send from Samsung S3 > On Jul 27, 2012 11:37 PM, "Alex Baranau" <alex.barano...@gmail.com> wrote: > > > -rwxr-xr-x 3 root root 1993369 2012-07-26 13:59 > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/S_T_MTX/1566523617482885717 > > > > "1993369" is the size. Oh sorry. It is 2MB, not 2GB. Yeah, that doesn't > > tell a lot. Looks like all data is in Memstore. As I said, you should try > > flushing the table, so that you can see where data was written. > > > > Of course it is always great to setup monitoring and see what is going on > > ;) > > > > Anyhow, the piece pasted above, means: > > > > table:SESSION_TIMELINE1, region: 0a5f6fadd0435898c6f4cf11daa9895a, > > columnFamily: S_T_MTX, hfile(created by memstore flush): > > 1566523617482885717, > > size: 1993369 bytes. > > > > btw, 2MB looks weird: very small flush size (in this case, in other cases > > this may happen - long story). May be compression does very well :) > > > > Alex Baranau > > ------ > > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch > - > > Solr > > > > On Fri, Jul 27, 2012 at 10:52 AM, syed kather <in.ab...@gmail.com> > wrote: > > > > > Alex Baranau, > > > > > > Can please tell how did you found it has 2GB of data from > > > "0a5f6fadd0435898c6f4cf11daa9895a" . I am pretty much intrested to know > > it > > > . > > > Thanks and Regards, > > > S SYED ABDUL KATHER > > > > > > > > > > > > On Fri, Jul 27, 2012 at 7:51 PM, Alex Baranau < > alex.barano...@gmail.com > > > >wrote: > > > > > > > From what you posted above, I guess one of the regions > > > > (0a5f6fadd0435898c6f4cf11daa9895a, > > > > note that it has 2 files 2GB each [1], while others regions are > > "empty") > > > is > > > > getting hit with writes. You may want to run "flush 'mytable'" > command > > > from > > > > hbase shell before looking at hdfs - this way you make sure your data > > is > > > > flushed to hdfs (and not hanged in Memstores). > > > > > > > > You may want to check the START/END keys of this region (via master > web > > > ui > > > > or in .META.). Then you can compare with the keys generated by your > > app. > > > > This should give you some info about what's going on. > > > > > > > > Alex Baranau > > > > ------ > > > > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - > > ElasticSearch > > > - > > > > Solr > > > > > > > > [1] > > > > > > > > -rwxr-xr-x 3 root root 1993369 2012-07-26 13:59 > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/S_T_MTX/1566523617482885717 > > > > > > > > -rwxr-xr-x 3 root root 2003372 2012-07-26 13:57 > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/S_T_MTX/7665015246030620502 > > > > > > > > On Fri, Jul 27, 2012 at 3:16 AM, Khang Pham <khang...@gmail.com> > > wrote: > > > > > > > > > Hi, > > > > > > > > > > by node do you mean regionserver node ? > > > > > > > > > > if you referring to RegionServer node: you can go to the hbase > master > > > web > > > > > interface master:65510/master.jsp to see load for each > regionserver. > > > > That's > > > > > the overall load. If you want to see load per node per table, you > > will > > > > need > > > > > to query on .META. table (column: info:server) > > > > > > > > > > > > > > > > > > > > --K > > > > > On Fri, Jul 27, 2012 at 9:07 AM, Mohit Anchlia < > > mohitanch...@gmail.com > > > > > >wrote: > > > > > > > > > > > Is there a way to see how much data does each node have per Hbase > > > > table? > > > > > > > > > > > > On Thu, Jul 26, 2012 at 5:53 PM, syed kather <in.ab...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > First check whether the data in hbase is consistent ... check > > this > > > by > > > > > > > running hbck (bin/hbase hbck ) If all the region is consistent > . > > > > > > > Now check no of splits in localhost:60010 for the table mention > > .. > > > > > > > On Jul 27, 2012 4:02 AM, "Mohit Anchlia" < > > mohitanch...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > I added new regions and the performance didn't improve. I > think > > > it > > > > > > still > > > > > > > is > > > > > > > > the load balancing issue. I want to ensure that my rows are > > > getting > > > > > > > > distrbuted accross cluster. What I see is this: > > > > > > > > > > > > > > > > Could you please tell me what's the best way to see the load? > > > > > > > > > > > > > > > > > > > > > > > > [root@dsdb4 ~]# hadoop fs -lsr /hbase/SESSION_TIMELINE1/ > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/8c02c8ed87e1a023ece8d8090a364641 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/8c02c8ed87e1a023ece8d8090a364641/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/8c02c8ed87e1a023ece8d8090a364641/.oldlogs/hlog.1343334723359 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/8c02c8ed87e1a023ece8d8090a364641/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/8c02c8ed87e1a023ece8d8090a364641/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/79e03d78a784601e8daa88aa85c39854 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/79e03d78a784601e8daa88aa85c39854/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/79e03d78a784601e8daa88aa85c39854/.oldlogs/hlog.1343334723093 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/79e03d78a784601e8daa88aa85c39854/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/79e03d78a784601e8daa88aa85c39854/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/1054fe861f199a23ebef8f49f99c4aba > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/1054fe861f199a23ebef8f49f99c4aba/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/1054fe861f199a23ebef8f49f99c4aba/.oldlogs/hlog.1343334723240 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/1054fe861f199a23ebef8f49f99c4aba/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/1054fe861f199a23ebef8f49f99c4aba/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/04295ef1d74887a107a013810f5aa26a > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/04295ef1d74887a107a013810f5aa26a/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/04295ef1d74887a107a013810f5aa26a/.oldlogs/hlog.1343334723171 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/04295ef1d74887a107a013810f5aa26a/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/04295ef1d74887a107a013810f5aa26a/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/486b20400be4a901d92ecded96d737cf > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/486b20400be4a901d92ecded96d737cf/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/486b20400be4a901d92ecded96d737cf/.oldlogs/hlog.1343334723397 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/486b20400be4a901d92ecded96d737cf/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 762 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/486b20400be4a901d92ecded96d737cf/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 4 2012-07-26 13:57 > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:59 > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/.tmp > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/.oldlogs/hlog.1343334723004 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 2 2012-07-26 13:59 > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 1993369 2012-07-26 13:59 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/S_T_MTX/1566523617482885717 > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 2003372 2012-07-26 13:57 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/S_T_MTX/7665015246030620502 > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 760 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/0a5f6fadd0435898c6f4cf11daa9895a/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/5eb02340b918e80e5018c908845a8495 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/5eb02340b918e80e5018c908845a8495/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/5eb02340b918e80e5018c908845a8495/.oldlogs/hlog.1343334723286 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/5eb02340b918e80e5018c908845a8495/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/5eb02340b918e80e5018c908845a8495/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/1f0735a7e08504357d0bca07e6772a75 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/1f0735a7e08504357d0bca07e6772a75/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/1f0735a7e08504357d0bca07e6772a75/.oldlogs/hlog.1343334723134 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/1f0735a7e08504357d0bca07e6772a75/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/1f0735a7e08504357d0bca07e6772a75/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/7ac1c8704c7d2e676b1dddd75c9b94f2 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/7ac1c8704c7d2e676b1dddd75c9b94f2/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/7ac1c8704c7d2e676b1dddd75c9b94f2/.oldlogs/hlog.1343334723323 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/7ac1c8704c7d2e676b1dddd75c9b94f2/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/7ac1c8704c7d2e676b1dddd75c9b94f2/.regioninfo > > > > > > > > > > > > > > > > drwxr-xr-x - root root 3 2012-07-26 13:32 > > > > > > > > /hbase/SESSION_TIMELINE1/14e92b7f58e521fd8c0b2d56cd01aaba > > > > > > > > > > > > > > > > drwxr-xr-x - root root 1 2012-07-26 13:32 > > > > > > > > > > > /hbase/SESSION_TIMELINE1/14e92b7f58e521fd8c0b2d56cd01aaba/.oldlogs > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 124 2012-07-26 13:32 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/14e92b7f58e521fd8c0b2d56cd01aaba/.oldlogs/hlog.1343334723204 > > > > > > > > > > > > > > > > drwxr-xr-x - root root 0 2012-07-26 13:32 > > > > > > > > > > /hbase/SESSION_TIMELINE1/14e92b7f58e521fd8c0b2d56cd01aaba/S_T_MTX > > > > > > > > > > > > > > > > -rwxr-xr-x 3 root root 764 2012-07-26 13:32 > > > > > > > > > > > > /hbase/SESSION_TIMELINE1/14e92b7f58e521fd8c0b2d56cd01aaba/.regioninfo > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Alex Baranau > > > > ------ > > > > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - > > ElasticSearch > > > - > > > > Solr > > > > > > > > > > > > > > > -- > > Alex Baranau > > ------ > > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch > - > > Solr > > > -- Alex Baranau ------ Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch - Solr
