[
https://issues.apache.org/jira/browse/METRON-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504697#comment-16504697
]
ASF GitHub Bot commented on METRON-1594:
----------------------------------------
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/1045
@nickwallen The PR has been up for 7 days, I addressed all community
comments days ago, and no comments appeared to be dissenting. Was there a
concern or issue you had before this was merged?
> KafkaWriter is asynchronous and may lose data on node failure
> -------------------------------------------------------------
>
> Key: METRON-1594
> URL: https://issues.apache.org/jira/browse/METRON-1594
> Project: Metron
> Issue Type: Bug
> Reporter: Michael Miklavcic
> Assignee: Michael Miklavcic
> Priority: Major
>
> Currently, we do not block for data to be sent from kafka writer and we do
> not batch. Unfortunately, the send call is asynchronous, so if the node dies
> before the data is actually sent from kafka then it will drop data. It is
> highly unlikely that we will be able to make kafkawriter synchronous in a
> performant way, so we will likely need to batch in the parser and enrichment
> topologies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
