[
https://issues.apache.org/jira/browse/S4-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189225#comment-13189225
]
Karthik Kambatla commented on S4-7:
-----------------------------------
The above discussed design has a minor issue with receiving ACKs from Netty and
dequeuing messages. This mostly arises from having a single
ChannelFutureListener for all the messages sent by the TCPEmitter. There is no
way to know for sure that the ACK corresponds to a particular message, I was
removing the head of the queue as it was the first one to be put on wire.
Minor changes to the design can fix this problem -
1. A SendQueue (per-partition) contains Messages.
2. A Message holds the byte[] and listens to the future of sending that
particular message.
I have implemented this new design and can upload the patch.
Should the patch include (1) only the changes after the currently uploaded
patch or (2) all changes including the ones that the uploaded patch covers.
> Netty to tolerate network glitches and connection loss
> ------------------------------------------------------
>
> Key: S4-7
> URL: https://issues.apache.org/jira/browse/S4-7
> Project: Apache S4
> Issue Type: Bug
> Reporter: Leo Neumeyer
> Assignee: Karthik Kambatla
> Fix For: 0.5
>
> Attachments: S4-7-Robust-TCPEmitter-asynchronous-ordered.patch
>
>
> NettyEmitter connects to different partitions and creates channels over which
> it communicates to other listeners.
> It suffers from the following issues --
> 1. If the underlying topology changes, the channels and the associated
> connections are not updated.
> 2. If a connection gets disconnected, it stays disconnected.
> 3. If for any reason, a connection can't be made, send() drops the message to
> be sent.
> The solution is to -
> 1. Maintain a bounded messageQueue for each destination partition - if a
> connection does not exist, the message should be queued.
> 2. Maintain a map of the channel used for each destination partition - update
> this map on changes to topology, or on send() in case of disconnections.
> 3. Every time a (re-)connection is made, send the queued messages first.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
