[
https://issues.apache.org/jira/browse/HBASE-19204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258038#comment-16258038
]
Chia-Ping Tsai edited comment on HBASE-19204 at 11/18/17 11:55 AM:
-------------------------------------------------------------------
It is blocked when closing {{HttpServer2}}. Zoom in on the {{HttpServer2}}
{code}
@Test
public void testHttpServerHang() throws Exception {
for (int i = 0; i != 500; ++i) {
HttpServer2 server = null;
try {
Configuration conf = new Configuration();
conf.setInt("hadoop.http.max.threads", 10);
server = createTestServer(conf);
server.addServlet("echo", "/echo", TestHttpServer.EchoServlet.class);
server.addServlet("echomap", "/echomap",
TestHttpServer.EchoMapServlet.class);
server.addServlet("htmlcontent", "/htmlcontent",
TestHttpServer.HtmlContentServlet.class);
server.addServlet("longheader", "/longheader",
TestHttpServer.LongHeaderServlet.class);
server.addJerseyResourcePackage(
JerseyResource.class.getPackage().getName(), "/jersey/*");
server.start();
} finally {
if (server != null) {
server.stop();
}
}
}
}
{code}
And then i got the following stack dump.
{quote}
"2020688994@qtp-1069352689-1 - Acceptor0
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39279" daemon
prio=10 tid=0x00007fbd8c7a4000 nid=0x1c3e waiting for monitor entry
[0x00007fbd740e1000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
- waiting to lock <0x00000000ec24c798> (a
org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
"main" prio=10 tid=0x00007fbd8c009000 nid=0x1bdc runnable [0x00007fbd93ad5000]
java.lang.Thread.State: RUNNABLE
at
org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:878)
- locked <0x00000000ec24c798> (a
org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
at
org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
- locked <0x00000000ec07da28> (a java.lang.Object)
at
org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
{quote}
And then i noticed the same observation is already in the HBASE-14430 and
HBASE-14505. [~stack] Do you still look for the cryptic JVM version? :)
was (Author: chia7712):
It is blocked when closing {{HttpServer2}}. Zoom in on the {{HttpServer2}}
{code}
@Test
public void testHttpServerHang() throws Exception {
for (int i = 0; i != 500; ++i) {
HttpServer2 server = null;
try {
Configuration conf = new Configuration();
conf.setInt("hadoop.http.max.threads", 10);
server = createTestServer(conf);
server.addServlet("echo", "/echo", TestHttpServer.EchoServlet.class);
server.addServlet("echomap", "/echomap",
TestHttpServer.EchoMapServlet.class);
server.addServlet("htmlcontent", "/htmlcontent",
TestHttpServer.HtmlContentServlet.class);
server.addServlet("longheader", "/longheader",
TestHttpServer.LongHeaderServlet.class);
server.addJerseyResourcePackage(
JerseyResource.class.getPackage().getName(), "/jersey/*");
server.start();
} finally {
if (server != null) {
server.stop();
}
}
}
}
{code}
And then i got the following stack dump.
{quote}
"2020688994@qtp-1069352689-1 - Acceptor0
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39279" daemon
prio=10 tid=0x00007fbd8c7a4000 nid=0x1c3e waiting for monitor entry
[0x00007fbd740e1000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
- waiting to lock <0x00000000ec24c798> (a
org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
"main" prio=10 tid=0x00007fbd8c009000 nid=0x1bdc runnable [0x00007fbd93ad5000]
java.lang.Thread.State: RUNNABLE
at
org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:878)
- locked <0x00000000ec24c798> (a
org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
at
org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
- locked <0x00000000ec07da28> (a java.lang.Object)
at
org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
{quote}
And then i noticed the same observation is already in the HBASE-14430 and
HBASE-14505. [~stack] Are you still look for the cryptic JVM version? :)
> branch-1.2 times out and is taking 6-7 hours to complete
> --------------------------------------------------------
>
> Key: HBASE-19204
> URL: https://issues.apache.org/jira/browse/HBASE-19204
> Project: HBase
> Issue Type: Umbrella
> Components: test
> Reporter: stack
>
> Sean has been looking at tooling and infra. This Umbrellas is about looking
> at actual tests. For example, running locally on dedicated machine I picked a
> random test, TestPerColumnFamilyFlush. In my test run, it wrote 16M lines. It
> seems to be having zk issues but it is catching interrupts and ignoring them
> ([~carp84] fixed this in later versions over in HBASE-18441).
> Let me try and do some fixup under this umbrella so we can get a 1.2.7 out
> the door.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
