[
https://issues.apache.org/jira/browse/S4-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175571#comment-13175571
]
Leo Neumeyer commented on S4-37:
--------------------------------
In some cases we need to provide guaranteed delivery, for example to train
models or for debugging, it is preferable to remove uncertainty. This means
that queues will need to block when they are full. Blocking should cascade all
te way to the adaptors. If the events are played back from the file system,
then the file reader should block until the incoming queue is ready.
To accomplish this we need acknowledgement and we need to implement stream
blocking even when the source and target PEs are in different nodes.
In production we may want certain events to have guaranteed delivery as well.
For example control events that have low volume but are used to coordinate
operations.
I assume this is all handled within the comm layer and there is no need for the
app to handle any of this, right?
> Emitters should allow registering a callback
> --------------------------------------------
>
> Key: S4-37
> URL: https://issues.apache.org/jira/browse/S4-37
> Project: Apache S4
> Issue Type: New Feature
> Affects Versions: 0.5
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 0.5
>
>
> In the current state, emitters (udp, tcp, and others) send the messages
> across. They do not necessarily acknowledge the information regarding the
> send (success/failure) to the calling layer.
> An ACK from the emitter is important to realize more complicated
> emitter/listener protocols. For instance, to implement a ReplicationEmitter
> (the message is sent to multiple nodes instead of one node, say for
> fault-tolerance), the message should be processed only after it is replicated
> the required number of times. So, if we are using TCP as the underlying
> layer, it should callback when the operation succeeds.
> I am eager to hear comments on (1) should we provision for complicated
> protocols? (2) if so, is there any better way of going about things?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
