[
https://issues.apache.org/jira/browse/HBASE-20777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521481#comment-16521481
]
Duo Zhang commented on HBASE-20777:
-----------------------------------
The same pattern, a new connection is established immediately after we stopping
the rpc server
{noformat}
2018-06-22 10:11:00,984 INFO [RS:0;asf911:40861] ipc.NettyRpcServer(143):
Stopping server on /67.195.81.155:40861
2018-06-22 10:11:01,013 INFO [RS-EventLoopGroup-3-6]
ipc.ServerRpcConnection(528): Auth successful for jenkins (auth:SIMPLE)
2018-06-22 10:11:01,016 INFO [RS-EventLoopGroup-3-6]
ipc.ServerRpcConnection(558): Connection from 67.195.81.155 port: 54547 with
version info: version: "3.0.0-SNAPSHOT" url:
"git://asf911.gq1.ygridcore.net/home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests"
revision: "98245ca6e4b6b8cffe22534f32e825c912de2ed3" user: "jenkins" date:
"Fri Jun 22 10:06:29 UTC 2018" src_checksum: "5a363d8828512f4a63f40412f9217b9e"
version_major: 3 version_minor: 0
{noformat}
{noformat}
2018-06-23 21:52:59,020 INFO [RS:1;asf911:60004] ipc.NettyRpcServer(143):
Stopping server on /67.195.81.155:60004
2018-06-23 21:52:59,050 INFO [RS-EventLoopGroup-5-23]
ipc.ServerRpcConnection(528): Auth successful for jenkins (auth:SIMPLE)
2018-06-23 21:52:59,051 INFO [RS-EventLoopGroup-5-23]
ipc.ServerRpcConnection(558): Connection from 67.195.81.155 port: 39762 with
version info: version: "3.0.0-SNAPSHOT" url:
"git://asf911.gq1.ygridcore.net/home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests"
revision: "55147c7eae784ebd9b4c5149112c6ec2623e4d48" user: "jenkins" date:
"Sat Jun 23 21:43:45 UTC 2018" src_checksum: "f9ff30ca9403247551ee80d0d87a9435"
version_major: 3 version_minor: 0
{noformat}
I guess the problem maybe that, a new connection is established while we
shutdown the rpc server and we miss to close the new connection, then it is
there forever. And if we use this connection, it will lead to rpc timeout...
> TestAsyncTableBatch is flakey
> -----------------------------
>
> Key: HBASE-20777
> URL: https://issues.apache.org/jira/browse/HBASE-20777
> Project: HBase
> Issue Type: Bug
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Major
> Attachments: HBASE-20777.patch,
> org.apache.hadoop.hbase.client.TestAsyncTableBatch-output.txt
>
>
> The log is very strange, we keep sending request to a dead RS, and the result
> is not connection refused, but rpc timeout, and later it becomes
> CallQueueTooBig...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
