Michael Miklavcic created METRON-1594: -----------------------------------------
Summary: KafkaWriter is asynchronous and may lose data on node failure Key: METRON-1594 URL: https://issues.apache.org/jira/browse/METRON-1594 Project: Metron Issue Type: Bug Reporter: Michael Miklavcic Assignee: Michael Miklavcic Currently, we do not block for data to be sent from kafka writer and we do not batch. Unfortunately, the send call is asynchronous, so if the node dies before the data is actually sent from kafka then it will drop data. It is highly unlikely that we will be able to make kafkawriter synchronous in a performant way, so we will likely need to batch in the parser and enrichment topologies. -- This message was sent by Atlassian JIRA (v7.6.3#76005)