Sandeep Nemuri created SLIDER-1161:
--------------------------------------
Summary: Improve regionserver status check in HBase Slider app
package
Key: SLIDER-1161
URL: https://issues.apache.org/jira/browse/SLIDER-1161
Project: Slider
Issue Type: Improvement
Components: app-package
Affects Versions: Slider 0.80
Environment: RHEL-6 (64 Bit)
Reporter: Sandeep Nemuri
*PROBLEM* :
Using slider for launching Hbase containers.
Following is the problem statement and details :
1. Assume region server went into a big pause and lost its heartbeat with
zookeeper
2. HMaster notices this and marks the region server as DEAD
3. However, slider agent continues to 'ps' the region server process in every
heartbeat.monitor.interval (45000ms in my case) and because it is just checking
for region server process being alive, it does not consider it dead
4. After that big delay, region server finally recovers and goes to HMaster
5. HMaster informs region server YouAreAlreadyDeadException
6. Now, this region server brings itself down and slider also notices that
process is no longer running.
7. Slider now launches a new region server.
The issue as clearly mentioned in steps above is that there can be a huge delay
between step 4 and 6. This means that we are now operating with lesser region
servers and this puts more and more load on existing region servers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
