[
https://issues.apache.org/jira/browse/CASSANDRA-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995281#comment-13995281
]
Joshua McKenzie commented on CASSANDRA-3569:
--------------------------------------------
I'm testing a v1 of this now - I removed the FD registration from StreamSession
entirely and split up the FD registration in RepairSession to be bounded around
Merkle-tree creation. Flow and failure work correctly on Windows but pulling a
network cable doesn't appear to be leaving my sockets half-open as the repair
sessions still die out with:
{code:title=misbehaving_sockets}
java.io.IOException: An established connection was aborted by the software in
your host machine
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
...
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:389)
[main/:na]
{code}
within about 30 seconds of me pulling the cable. I have the KeepAliveTime reg
key set to 300000 so it's either not respecting that or the TCP connection is
getting forcibly killed elsewhere in the system.
I'll test it on linux to see if I get some more reliable test results.
> Failure detector downs should not break streams
> -----------------------------------------------
>
> Key: CASSANDRA-3569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3569
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Peter Schuller
> Assignee: Joshua McKenzie
> Fix For: 2.1 rc1
>
> Attachments: 3569-2.0.txt
>
>
> CASSANDRA-2433 introduced this behavior just to get repairs to don't sit
> there waiting forever. In my opinion the correct fix to that problem is to
> use TCP keep alive. Unfortunately the TCP keep alive period is insanely high
> by default on a modern Linux, so just doing that is not entirely good either.
> But using the failure detector seems non-sensicle to me. We have a
> communication method which is the TCP transport, that we know is used for
> long-running processes that you don't want to incorrectly be killed for no
> good reason, and we are using a failure detector tuned to detecting when not
> to send real-time sensitive request to nodes in order to actively kill a
> working connection.
> So, rather than add complexity with protocol based ping/pongs and such, I
> propose that we simply just use TCP keep alive for streaming connections and
> instruct operators of production clusters to tweak
> net.ipv4.tcp_keepalive_{probes,intvl} as appropriate (or whatever equivalent
> on their OS).
> I can submit the patch. Awaiting opinions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
