[
https://issues.apache.org/jira/browse/HBASE-24206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091574#comment-17091574
]
Peter Somogyi commented on HBASE-24206:
---------------------------------------
What I have managed to dig up from the logs is the following.
There was a SCP that initiated 25 AssignProcedures from which 3 did not finish
and ended up in a STUCK state. In the upcoming days there were multiple SCPs.
It looks like the original SCP was still holding the lock for these 3 regions.
AssignProcedure failed since it was not able to acquire the lock and started a
rollback. Since the rollback of a SCP throws UnsupportedOperationException the
CODE-BUG message was logged.
My initial idea was that we can just ignore this CODE-BUG for
UnsupportedOperationException but it might be something bigger as you and
[~zhangduo] mentioned and this would just hide the problem.
> Unsupported rollback should not be logged as CODE-BUG
> ------------------------------------------------------
>
> Key: HBASE-24206
> URL: https://issues.apache.org/jira/browse/HBASE-24206
> Project: HBase
> Issue Type: Improvement
> Components: logging, proc-v2
> Reporter: Peter Somogyi
> Assignee: Peter Somogyi
> Priority: Minor
>
> When executing rollback from a state that does not support rollback
> functionality UnsupportedOperationException is thrown. The Procedure
> framework does not handle this case specially and logs an ERROR message
> because it is caught as a Throwable.
> This scenario should have a cleaner log message.
> {noformat}
> 2020-04-08 20:57:10,137 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception for pid=14791, state=FAILED:SERVER_CRASH_START,
> locked=true,
> exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
> AssignProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
> 6f48ff190e182c204056eb8b227ff6e8 owned by pid=9419, CANNOT run 'this'
> (pid=14797).; ServerCrashProcedure server=example.com,16020,1584086541944,
> splitWal=true, meta=false
> java.lang.UnsupportedOperationException: unhandled
> state=SERVER_CRASH_GET_REGIONS
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.rollbackState(ServerCrashProcedure.java:265)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.rollbackState(ServerCrashProcedure.java:59)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:982)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1663)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1595)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1438)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1200(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2058)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
