[
https://issues.apache.org/jira/browse/HBASE-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340711#comment-16340711
]
Appy commented on HBASE-19866:
------------------------------
As for the root cause, i see that backup master is not stopping.
bq. Process Thread Dump: Automatic Stack Trace every 60 seconds waiting on
M:0;b10d9a4963f1:49296
M:0;b10d9a4963f1:49296 is backup HMaster thread
bq. 2018-01-25 12:52:52,355 INFO [M:0;b10d9a4963f1:49296]
master.HMaster(2006): Adding backup master ZNode
/hbase/backup-masters/b10d9a4963f1,49296,1516884770433
And i think it's stuck on NettyRpcServer?
{noformat}
Thread 209 (M:0;b10d9a4963f1:49296):
State: WAITING
Blocked count: 164
Waited count: 334
Waiting on
org.apache.hbase.thirdparty.io.netty.channel.group.DefaultChannelGroupFuture@7b59ef17
Stack:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:254)
org.apache.hbase.thirdparty.io.netty.channel.group.DefaultChannelGroupFuture.awaitUninterruptibly(DefaultChannelGroupFuture.java:183)
org.apache.hbase.thirdparty.io.netty.channel.group.DefaultChannelGroupFuture.awaitUninterruptibly(DefaultChannelGroupFuture.java:40)
org.apache.hadoop.hbase.ipc.NettyRpcServer.stop(NettyRpcServer.java:148)
org.apache.hadoop.hbase.regionserver.RSRpcServices.stop(RSRpcServices.java:1405)
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1134)
org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:570)
java.lang.Thread.run(Thread.java:748)
{noformat}
Ping [~aoxiang] [~stack]
> TestRegionServerReportForDuty doesn't timeout
> ---------------------------------------------
>
> Key: HBASE-19866
> URL: https://issues.apache.org/jira/browse/HBASE-19866
> Project: HBase
> Issue Type: Bug
> Reporter: Appy
> Assignee: Appy
> Priority: Major
>
> So reading around junit docs, looks like the reason is result of these two
> rules:
> - @Test(timeout=X) applies only on the test function, and not on whole test
> fixture (@After, @Before, etc)
> - Timeout rule applies on whole test fixture
> TestRegionServerReportForDuty just has @Test(timeout=180000) and no Timeout
> rule unlike we have in so many other tests.
> The test method, in the logs I have, runs in less then 60 sec. So it meets
> the timeout specified in @Test annotation.
> However, we get stuck in tearDown, and since there is no Timeout rule, it
> keeps on running until surefire kills the JVM after
> forkedProcessTimeoutInSeconds (set to 900 sec).
> Let use the "Timeout" rule instead of {{@Test(timeout=180000)}}.
> *However, note that this won't solve the root cause of hangup.* It'll just
> make the test fail neatly rather than getting stuck and requiring surefire
> plugin to kill the forked JVMs (see HBASE-19803).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
