Hello Members, I am using Hbase 2.4.4 and facing some issues,
1: hbase:meta region is getting stuck at opening state while restarting Hbase cluster. Symptoms: A few of the region servers and Master nodes went down, including one which was holding the hbase:meta region. After restarting, in logs I am seeing a holding pattern until region online with Server Crash Procedure(SCP) as true. I waited for 30 mins but the procedure was still stuck. Actions performed. 1: Bypass all stuck procedures using HBCK2. 2: Try to assign hbase:meta from HBCK2. After this new procedure ID got created but still it was stuck in the Opening state. 3: Try to set region state OPEN and CLOSED and restart master but no luck on meta transition state. Finally stopped all region servers and masters, then started only one master and one region server, in which meta was previously hosted. And after assigning from HBCK2, meta got assigned successfully, further before assigning hbase:namespace, started all other region servers and then assigned namespace as well after this Hbase cluster came up. Observation is that hbase:meta is only coming up in the same region server where it was hosted before restart. This is the first time I have seen this behaviour. Seeking help if someone faced issues where the meta region is stuck in the opening state, and how can I further debug why it is getting stuck in the opening state . Hbase region balancer was on during this activity. Regards, Chetan
