Sounds like your access patterns are not balanced well, you have a hotspot. Have a look at the metrics emitted from that machine. It will tell you which region is winning the popularity contest.
On Monday, December 15, 2014, uamadman <uamadm...@gmail.com> wrote: > I've done a bit of digging and hope someone can shed some light on my > particular issue. One and Only One of my region servers after each restart > is randomly "Plagued" with a single maxed out CPU-Core and a Read Request > chart registering around 40k read requests per second. The remaining 13 > dance around 2-5% and sit promptly at 0 reads per/s. Is this normal? And > Why? > > Cluster Stats: 1 Master/14 Nodes Quad\Dual-Core 2.8-3.0Ghz 6-8gb ram each > with a single 500Gb Drive > Table Stats: ~150 Tables Total Split into 590 Regions > 1 table w/ 2.5Billion Rows 500 Columns > 4 tables w/ 60-250 Million Rows 1000 Columns > ~145 tables w/ 100K Rows 25 Columns > > Standard-Key-Template: abcd123456 > > Versions: > > CDH: 5.1.2-1.cdh5.1.2.p0.3 > > HBase Version: 0.98.1-cdh5.1.2, rUnknown > > HBase Compiled: Mon Aug 25 19:33:59 PDT 2014, jenkins > > Hadoop Version: 2.3.0-cdh5.1.2, r8e266e052e423af592871e2dfe09d54c03f6a0e8 > > Hadoop Compiled: 2014-08-26T01:36Z, jenkins > > Cheers, > Jon > > > > -- > View this message in context: > http://apache-hbase.679495.n3.nabble.com/Region-Server-Thread-with-a-Single-High-Idle-CPU-tp4066876.html > Sent from the HBase User mailing list archive at Nabble.com. >
