Alessandro D'Armiento created NIFI-8130:
-------------------------------------------

             Summary: PutDatabaseRecord after MergeRecord randomly hangs 
forcing to discard the whole queue
                 Key: NIFI-8130
                 URL: https://issues.apache.org/jira/browse/NIFI-8130
             Project: Apache NiFi
          Issue Type: Bug
            Reporter: Alessandro D'Armiento


This bug is hard to replicate as it happens randomly.

In the following (common) configuration, in which multiple records are merged 
and then sent to a PutDatabaseRecord, it happens sometimes that a specific 
FlowFile cause the PutDatabaseRecord to fail with `FlowFileHandlingException: 
FlowFile already marked for transfer`

!Screenshot 2021-01-11 at 11.38.45.png!

!Screenshot 2021-01-11 at 11.34.06.png!

In case of such an event, the processor remains stuck trying to process that 
specific FlowFile (i.e. it is not routed to the failure relationship). This 
forces the user to empty the whole queue in order to continue, which causes 
data loss. 

I noticed the following: 
 * The issue is bound with the FlowFile: the same FlowFile will make multiple 
processors to fail with the same error.
 * Creating a new FlowFile with the same content (i.e. publishing the FlowFile 
on a Kafka queue and consuming it right after) doesn't solve the issue, and the 
FlowFile will raise the error again once sent to the PutDatabaseRecord
 * This error happened to me only when using the PutDatabaseRecord after a 
MergeRecord (in order to batch multiple records in a single DB transaction). 
 * This issue was already raised in the Cloudera Community Forum, alas, without 
any answer. 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to