[
https://issues.apache.org/jira/browse/S4-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204756#comment-13204756
]
Karthik Kambatla commented on S4-7:
-----------------------------------
Indeed as expected, the issue is with storing the nodes as a list in the
Cluster.
In the multiple partition scenario, a ClusterNode created with partitionId i is
not necessarily stored at the i'th position in the list. I confirmed this by
adding a check in TCPEmitter.
Observed run with synchronization issues -
- ERROR org.apache.s4.comm.tcp.TCPEmitter - FATAL: ClusterNode 4 is stored as 0
- Subsequently, ReceiveThread 0 doesn't receive any messages from 1, and
ReceiveThread 4 receives twice as many from 1.
I ll modify Cluster to use HashBiMap instead of List and hopefully that would
fix all synchronization issues.
> Netty to tolerate network glitches and connection loss
> ------------------------------------------------------
>
> Key: S4-7
> URL: https://issues.apache.org/jira/browse/S4-7
> Project: Apache S4
> Issue Type: Bug
> Reporter: Leo Neumeyer
> Assignee: Karthik Kambatla
> Fix For: 0.5
>
> Attachments: S4-7-Robust-TCPEmitter-asynchronous-ordered.patch,
> s4-7.patch, s4-7.patch
>
>
> NettyEmitter connects to different partitions and creates channels over which
> it communicates to other listeners.
> It suffers from the following issues --
> 1. If the underlying topology changes, the channels and the associated
> connections are not updated.
> 2. If a connection gets disconnected, it stays disconnected.
> 3. If for any reason, a connection can't be made, send() drops the message to
> be sent.
> The solution is to -
> 1. Maintain a bounded messageQueue for each destination partition - if a
> connection does not exist, the message should be queued.
> 2. Maintain a map of the channel used for each destination partition - update
> this map on changes to topology, or on send() in case of disconnections.
> 3. Every time a (re-)connection is made, send the queued messages first.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
