Ron Buckley created HBASE-6905:
----------------------------------
Summary: Regionserver Metrics reporting as negative numbers
Key: HBASE-6905
URL: https://issues.apache.org/jira/browse/HBASE-6905
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Ron Buckley
Priority: Minor
Our HBase cluster has been up since May. Over the weekend one of our monitors
started barking because the readRequestsCount is being reported as negative.
requestsPerSecond=17, numberOfOnlineRegions=81, numberOfStores=91,
numberOfStorefiles=115, storefileIndexSizeMB=2, rootIndexSizeKB=2497,
totalStaticIndexSizeKB=197243, totalStaticBloomSizeKB=218013,
memstoreSizeMB=286, readRequestsCount=-1820030847,
writeRequestsCount=326458652, compactionQueueSize=0, flushQueueSize=0,
usedHeapMB=6303, maxHeapMB=10214, blockCacheSizeMB=4137.7,
blockCacheFreeMB=969.52, blockCacheCount=60445, blockCacheHitCount=3040378952,
blockCacheMissCount=501619888, blockCacheEvictedCount=134409709,
blockCacheHitRatio=85%, blockCacheHitCachingRatio=95%,
hdfsBlocksLocalityIndex=100
These are the current counts our regiion servers are reporting.
hadoopsup@wchddb01pxdu:~/hbaseTools$ ./getHBaseMetric.sh readRequestsCount |
sort -k +2 -t = -n
wchddb41pxdu.prod.oclc.org:9004 readRequestsCount=-2140115894
wchddb18pxdu.prod.oclc.org:9004 readRequestsCount=-2129290048
wchddb33pxdu.prod.oclc.org:9004 readRequestsCount=-1938060269
wchddb22pxdu.prod.oclc.org:9004 readRequestsCount=-1819997297
wchddb37pxdu.prod.oclc.org:9004 readRequestsCount=-1361713407
wchddb39pxdu.prod.oclc.org:9004 readRequestsCount=-1346438303
wchddb15pxdu.prod.oclc.org:9004 readRequestsCount=23724328
wchddb17pxdu.prod.oclc.org:9004 readRequestsCount=26673900
wchddb31pxdu.prod.oclc.org:9004 readRequestsCount=26753467
wchddb08pxdu.prod.oclc.org:9004 readRequestsCount=29632942
wchddb50pxdu.prod.oclc.org:9004 readRequestsCount=42054983
wchddb43pxdu.prod.oclc.org:9004 readRequestsCount=91783220
wchddb45pxdu.prod.oclc.org:9004 readRequestsCount=276019000
wchddb12pxdu.prod.oclc.org:9004 readRequestsCount=332978804
wchddb14pxdu.prod.oclc.org:9004 readRequestsCount=414414026
wchddb25pxdu.prod.oclc.org:9004 readRequestsCount=714715497
wchddb46pxdu.prod.oclc.org:9004 readRequestsCount=854608441
wchddb28pxdu.prod.oclc.org:9004 readRequestsCount=1056464575
wchddb40pxdu.prod.oclc.org:9004 readRequestsCount=1142317805
wchddb11pxdu.prod.oclc.org:9004 readRequestsCount=1192703412
wchddb07pxdu.prod.oclc.org:9004 readRequestsCount=1324556090
wchddb23pxdu.prod.oclc.org:9004 readRequestsCount=1345624237
wchddb47pxdu.prod.oclc.org:9004 readRequestsCount=1354096261
wchddb38pxdu.prod.oclc.org:9004 readRequestsCount=1494019278
wchddb27pxdu.prod.oclc.org:9004 readRequestsCount=1556667925
wchddb10pxdu.prod.oclc.org:9004 readRequestsCount=1578493607
wchddb21pxdu.prod.oclc.org:9004 readRequestsCount=1621338673
wchddb29pxdu.prod.oclc.org:9004 readRequestsCount=1623831234
wchddb48pxdu.prod.oclc.org:9004 readRequestsCount=1643042108
wchddb24pxdu.prod.oclc.org:9004 readRequestsCount=1662372209
wchddb34pxdu.prod.oclc.org:9004 readRequestsCount=1709785225
wchddb20pxdu.prod.oclc.org:9004 readRequestsCount=1710768413
wchddb42pxdu.prod.oclc.org:9004 readRequestsCount=1710792044
wchddb30pxdu.prod.oclc.org:9004 readRequestsCount=1742485411
wchddb26pxdu.prod.oclc.org:9004 readRequestsCount=1819557634
wchddb49pxdu.prod.oclc.org:9004 readRequestsCount=1856456352
wchddb44pxdu.prod.oclc.org:9004 readRequestsCount=1858474961
wchddb06pxdu.prod.oclc.org:9004 readRequestsCount=1894403007
wchddb13pxdu.prod.oclc.org:9004 readRequestsCount=1938887475
wchddb09pxdu.prod.oclc.org:9004 readRequestsCount=1970495619
wchddb05pxdu.prod.oclc.org:9004 readRequestsCount=1998165666
wchddb16pxdu.prod.oclc.org:9004 readRequestsCount=2003054336
wchddb04pxdu.prod.oclc.org:9004 readRequestsCount=2015847806
wchddb32pxdu.prod.oclc.org:9004 readRequestsCount=2134753437
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
