Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

bq. there is a bulkPut available in Spark-HBase module The bulkPut is abstraction for HBaseContext.foreachPartition method. Please take a look at hbaseBulkLoad() in HBaseRDDFunctions. Note: if you use Sparl 2.0, you would need the latest patch on HBASE-16179. Cheers On Thu, Sep 22, 2016 at

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

Thanks Ted and Vlad I upload the screenshots to flickr, seems they are more clear there 1. MutateCount(Put Count) at table level(null as gap) https://flic.kr/p/LxLa6d 2. MutateCount(Put Count) at table level(null as line

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

For tuning G1GC, see the head of https://blogs.apache.org/hbase/ FYI On Thu, Sep 22, 2016 at 10:30 AM, Vladimir Rodionov wrote: > Your RS was declared dead because of a long GC. > > What you can do: > > 1. Tweak CMS config: -XX:CMSInitiatingOccupancyFraction=50 >

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

Your RS was declared dead because of a long GC. What you can do: 1. Tweak CMS config: -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSInitiatingOccupancyFractionOnly. Plus increase heap size accordingly to accommodate decreasing of a working set size (now CMS starts when 50% of heap is occupied).

Re: [RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

bq. the MutateCount metrics didn't show there is a peak time before and after the problem on that table In the graph, there was a purple line with spikes - I assume this was not for the table with max MemStoreSize. The legend in the graph is hard to read. If the sizes of the Puts vary, that may

[RegionServer Dead] Identify HBase Table Cause RegionServer Dead(Version 1.0.0-cdh5.5.2)

Hi, HBase Users I encounter a RegionServer Dead case, and want to identify which HBase Table actually cause the problem Based on HBase JMX Restful Service, what I have on hand is: 1. Table level Grafana monitoring for Put, Get, Scan, MemStoreSize 2. RegionServer level Grafana monitoring for GC