[
https://issues.apache.org/jira/browse/HBASE-26283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17414839#comment-17414839
]
bolao commented on HBASE-26283:
-------------------------------
** threre are some logs for this case, we can found tow procedure to
assign the same region，the previous procedure open region successfully，the new
procedure will open region again，and stuck in RIT。why the region will get more
than one procedure, i still don't know.
*old procedure***
2021-09-09 14:08:28 [PEWorker-27] INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.waitRegions(737)
-Took xlock for pid=374353, ppid=366849,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=NB_APP_ADDRESS_ALL_000000,
region=1d76a0df861cd8063318a776e0b8532a, ASSIGN
2021-09-09 14:08:28 [PEWorker-27] INFO
org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.queueAssign(194)
-Starting pid=374353, ppid=366849,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true;
TransitRegionStateProcedure table=NB_APP_ADDRESS_ALL_000000,
region=1d76a0df861cd8063318a776e0b8532a, ASSIGN; state=OPEN, location=null;
forceNewPlan=true, retain=false、
2021-09-09 14:09:55 [PEWorker-15] INFO
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(1704)
-Initialized subprocedures=[\{pid=426382, ppid=374353, state=RUNNABLE;
OpenRegionProcedure 1d76a0df861cd8063318a776e0b8532a,
server=fx-hd-sc-hbase-slave-3.fx-hd-sc.fx-ns.svc.cluster.sxxz,16020,1631097275542}]
*new Procedure*
2021-09-09 14:09:21 [PEWorker-9] INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.waitRegions(728)
-Waiting on xlock for pid=418061,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
TransitRegionStateProcedure table=NB_APP_ADDRESS_ALL_000000,
region=1d76a0df861cd8063318a776e0b8532a, ASSIGN held by pid=374353
2021-09-09 14:09:55 [PEWorker-15] INFO
org.apache.hadoop.hbase.master.assignment.RegionStateStore.updateUserRegionLocation(216)
-pid=418061 updating hbase:meta row=1d76a0df861cd8063318a776e0b8532a,
regionState=OPENING,
regionLocation=fx-hd-sc-hbase-slave-3.fx-hd-sc.fx-ns.svc.cluster.sxxz,16020,1631097275542
*RegionServer*
2021-09-09 14:09:56
[RS_OPEN_REGION-regionserver/fx-hd-sc-hbase-slave-3:16020-3] INFO
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(2285)
-Post open deploy tasks for
NB_APP_ADDRESS_ALL_000000,D\x180600\x18,1436985083592.1d76a0df861cd8063318a776e0b8532a.,
pid=426382, masterSystemTime=1631167796095
2021-09-09 14:09:56
[RS_OPEN_REGION-regionserver/fx-hd-sc-hbase-slave-3:16020-3] INFO
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(145)
-Opened
NB_APP_ADDRESS_ALL_000000,D\x180600\x18,1436985083592.1d76a0df861cd8063318a776e0b8532a.
> RegionServer ignored a procedure dute to repeatedly opening which lead region
> stuck in RIT
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-26283
> URL: https://issues.apache.org/jira/browse/HBASE-26283
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Affects Versions: 2.3.5
> Reporter: bolao
> Priority: Minor
>
> when hbase cluster shut down Suddenlly, there are some repeatedly
> opening regions and cause them to stuck in RTI。we found
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler#process()
> directly return when handle repeatedly opening regions， and don't report to
> master, master unable to process next status for this procedure， and
> regions stuck in RIT。so can we do like this:
> # AssignRegionHandler report the region is online if regionserver found the
> region is online,
> # or AssignRegionHandler report the region is failed online if regionserver
> found the region is online, let Hmater to retry。
> please give some suggestions, thanks !
> {code:java}
> // code placeholder
> Region onlineRegion = rs.getRegion(encodedName); if (onlineRegion != null) {
> LOG.warn("Received OPEN for the region:{}, which is already online",
> regionName);
> // Just follow the old behavior, do we need to call
> reportRegionStateTransition? Maybe not?
> // For normal case, it could happen that the rpc call to schedule this
> handler is succeeded,
> // but before returning to master the connection is broken. And when master
> tries again, we
> // have already finished the opening. For this case we do not need to call
> // reportRegionStateTransition any more.
> return; }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
