Mark Payne created NIFI-2865: -------------------------------- Summary: Address issues of PublishKafka blocking when having trouble communicating with Kafka broker and improve performance Key: NIFI-2865 URL: https://issues.apache.org/jira/browse/NIFI-2865 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.1.0
When NiFi is unable to communicate properly with the Kafka broker, we see the NiFi threads occasionally block. This should be resolvable by calling the wakeup() method of the client. Additionally, if Kafka takes too long to respond, we should be able to route the FlowFile to failure and move on. PublishKafka has a nice feature that allows a demarcated stream to be sent as separate messages, so that a large number of messages can be sent as a single FlowFile. However, in the case of individual messages per FlowFile, the performance could be improved by batching together multiple FlowFiles per session -- This message was sent by Atlassian JIRA (v6.3.4#6332)