[
https://issues.apache.org/jira/browse/APEXMALHAR-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557187#comment-15557187
]
Sandesh commented on APEXMALHAR-2283:
-------------------------------------
During recovery - 0.9 Kafka Output operator.
appId + operatorId -> used to distinguish tuples from other writers
Extra window state -> tracks Kafka offsets, tuples between that Kafka offsets
are read during recovery. To construct partial window.
Hashcode -> replayed tuples are dropped rather than writing to Kafka
It is important to make sure that logic works, as we have few customers using
this operator in production.
> Refactor kafka output operator
> ------------------------------
>
> Key: APEXMALHAR-2283
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2283
> Project: Apache Apex Malhar
> Issue Type: Improvement
> Reporter: Siyuan Hua
> Assignee: Siyuan Hua
>
> The abstract kafka output operator needs to be refactored
> 1. Needs to set some mandatory properties on operator level instead of kafka
> property level.
> 2. More document and examples
> 3. Find a standard way to achieve exactly once in both 0.8 and 0.9
> More will be added when working on the ticket
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
