[
https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730278#comment-14730278
]
Akihiro Suda commented on ZOOKEEPER-2251:
-----------------------------------------
Hi [~nijel], thank you for the information.
The
[{{wait()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515]
might be waiting for this
[{{notifyAll()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxn.java#L736].
A typical call stack for the notifyAll() is as follows:
- {{Object.notifyAll()}}
-
[{{ClientCnxn.finishPacket()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxn.java#L736]
-
[{{ClientCnxn$SendThread.readResponse()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxn.java#L948]
-
[{{ClientCnxnSocketNIO.doIO()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxnSocketNIO.java#L99]
-
[{{ClientCnxnSocketNIO.doTransport()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxnSocketNIO.java#L361]
-
[{{ClientCnxn$SendThread.run()}}|https://github.com/apache/zookeeper/blob/7f10f9c53a48b296dd27c7a104b303b10b045a89/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1236]
If you can still reproduce the bug, could you please add some logs for the
above methods and share the result (plus ZooKeeper version and YARN version)?
Thanks
> Add Client side packet response timeout to avoid infinite wait.
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-2251
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251
> Project: ZooKeeper
> Issue Type: Bug
> Reporter: nijel
> Assignee: Arshad Mohammad
>
> I came across one issue related to Client side packet response timeout In my
> cluster many packet drops happened for some time.
> One observation is the zookeeper client got hanged. As per the thread dump it
> is waiting for the response/ACK for the operation performed (synchronous API
> used here).
> I am using
> zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory
> Since only few packets missed there is no DISCONNECTED event occurred.
> Need add a "response time out" for the operations or packets.
> *Comments from [~rakeshr]*
> My observation about the problem:-
> * Can use tools like 'Wireshark' to simulate the artificial packet loss.
> * Assume there is only one packet in the 'outgoingQueue' and unfortunately
> the server response packet lost. Now, client will enter into infinite
> waiting.
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515
> * Probably we can discuss more about this problem and possible solutions(add
> packet ACK timeout or another better approach) in the jira.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
