[hbase] branch branch-2.5 updated: HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction (#4383) (#4409)

[huaxiangsun]

This is an automated email from the ASF dual-hosted git repository.

huaxiangsun pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2.5 by this push:
     new c337e391e2 HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos 
GracefulRollingRestartRsAction (#4383) (#4409)
c337e391e2 is described below

commit c337e391e25d28f2f667e465bdae78048158417d
Author: huaxiangsun <huaxiang...@apache.org>
AuthorDate: Thu May 5 13:43:33 2022 -0500

    HBASE-26984 Chaos Monkey thread dies in ITBLL Chaos 
GracefulRollingRestartRsAction (#4383) (#4409)
    
    There are two cases here:
      1. Chaos Monkey thread died and there is no chaos after that.
      2. Sometimes, regions are being moved back too quick that region server 
has not finished its initliazation yet.
         wait sometime to make sure that region server finishes its 
initialization.
    
    Signed-off-by:  Wellington Chevreuil <wellington.chevre...@gmail.com>
---
 .../hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java      | 2 ++
 .../org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java  | 2 ++
 2 files changed, 4 insertions(+)

diff --git 
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
 
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
index 236ed21290..6fe6332813 100644
--- 
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
+++ 
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/GracefulRollingRestartRsAction.java
@@ -63,6 +63,8 @@ public class GracefulRollingRestartRsAction extends 
RestartActionBaseAction {
         rm.load();
       } catch (Shell.ExitCodeException e) {
         getLogger().info("Problem restarting but presume successful; code={}", 
e.getExitCode(), e);
+      } catch (Exception e) {
+        getLogger().info("Exception but continuing...", e);
       }
       sleep(rand.nextInt((int) sleepTime));
     }
diff --git 
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
 
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
index 31e9f578b9..3edb1fabdd 100644
--- 
a/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
+++ 
b/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartActionBaseAction.java
@@ -80,6 +80,8 @@ public abstract class RestartActionBaseAction extends Action {
     sleep(sleepTime);
     getLogger().info("Starting region server: {}", server);
     startRs(server);
+    // Sleep some time to make sure RS is online.
+    sleep(sleepTime);
   }
 
   void restartZKNode(ServerName server, long sleepTime) throws IOException {