zhaoyunjiong created HBASE-12667: ------------------------------------ Summary: Deadlock in AssignmentManager Key: HBASE-12667 URL: https://issues.apache.org/jira/browse/HBASE-12667 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: zhaoyunjiong
No order between regionPlans and regionStates caused dead lock. Trunk don't have the problem since it's already got refactor. "master:phxhshdc11en0004:60000": at org.apache.hadoop.hbase.master.AssignmentManager.clearRegionPlan(AssignmentManager.java:2898) - waiting to lock <0x000000048cefe520> (a java.util.TreeMap) at org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:1286) at org.apache.hadoop.hbase.master.AssignmentManager.handleRegionSplitting(AssignmentManager.java:3552) - locked <0x000000048cf6fc10> (a org.apache.hadoop.hbase.master.RegionStates) at org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:732) at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:601) at org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRecoverLostRegions(AssignmentManager.java:2851) at org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:519) at org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:459) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:900) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:609) at java.lang.Thread.run(Thread.java:744) "AM.-pool1-t10": at org.apache.hadoop.hbase.master.RegionStates.getRegionAssignments(RegionStates.java:154) - waiting to lock <0x000000048cf6fc10> (a org.apache.hadoop.hbase.master.RegionStates) at org.apache.hadoop.hbase.master.AssignmentManager.getSnapShotOfAssignment(AssignmentManager.java:3610) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.getRegionAssignmentsByServer(BaseLoadBalancer.java:1146) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:959) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1010) at org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:2209) - locked <0x000000048cefe520> (a java.util.TreeMap) at org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:2166) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1886) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1445) at org.apache.hadoop.hbase.master.AssignCallable.call(AssignCallable.java:45) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.3.4#6332)