[
https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15055604#comment-15055604
]
Akihiro Suda commented on ZOOKEEPER-2251:
-----------------------------------------
Hi [~arshad.mohammad], thank you for the response.
The attached patch assumes that a response packet can be "lost".
But I'm still not sure what "lost" means.
https://issues.apache.org/jira/browse/ZOOKEEPER-2251?focusedCommentId=14706468&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14706468
If TCP packets are actually lost, Linux kernel (on the server) should
retransmit packets.
If even retransmitted packets are lost, client kernel should raise
{{ETIMEDOUT}} after 924.6 seconds by default. (It depends on the value of
{{/proc/sys/net/ipv4/tcp_retries2}}.)
So I think infinite wait should not happen even if packets are lost.
I can come up with following possibilities:
1. Client kernel raised {{ETIMEDOUT}}, and JVM raised a corresponding
{{java.lang.IOException}}. But client ZK did not catch the {{IOException}}
correctly. It's a bug of client.
2. Server sent a TCP FIN packet. Client received the TCP FIN packet and raised
{{IOException}}, but client ZK did not catch the {{IOException}} correctly.
It's a bug of client.
3. No TCP FIN packet were sent. It's a bug of server.
4. Bug of kernel or JVM.
I think we should dig out what really happens when we hit the bug, before
merging the patch and closing the JIRA.
(No matter what is the real cause of the bug, the patch is useful! Because
non-privileged users cannot modify {{/proc/sys/net/ipv4/*}}.)
I would like to hear your comments.
I'm also trying to reproduce the bug again, and digging out what are really
happening.
Thanks!
> Add Client side packet response timeout to avoid infinite wait.
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-2251
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251
> Project: ZooKeeper
> Issue Type: Bug
> Reporter: nijel
> Assignee: Arshad Mohammad
> Attachments: ZOOKEEPER-2251-01.patch, ZOOKEEPER-2251-02.patch,
> ZOOKEEPER-2251-03.patch
>
>
> I came across one issue related to Client side packet response timeout In my
> cluster many packet drops happened for some time.
> One observation is the zookeeper client got hanged. As per the thread dump it
> is waiting for the response/ACK for the operation performed (synchronous API
> used here).
> I am using
> zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory
> Since only few packets missed there is no DISCONNECTED event occurred.
> Need add a "response time out" for the operations or packets.
> *Comments from [~rakeshr]*
> My observation about the problem:-
> * Can use tools like 'Wireshark' to simulate the artificial packet loss.
> * Assume there is only one packet in the 'outgoingQueue' and unfortunately
> the server response packet lost. Now, client will enter into infinite
> waiting.
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515
> * Probably we can discuss more about this problem and possible solutions(add
> packet ACK timeout or another better approach) in the jira.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
