Hi, I came across an issue, which looks similar to https://stackoverflow.com/questions/51891413/nifi-processor-task-count-is-very-high-what-must-be-the-reason Many tasks/time on downstream processor.
In my case, we are using PutElasticsearchHttpRecord and the issue is, when RecordReader cannot parse input records (e.g. invalid JSON). We fall into catch block: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java#L455 Where the FlowFile is penalized and transferred to FAILURE. Now the questions: 1. is penalization meant for retry-able errors? According to https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#penalization-vs-yielding it seems like the answer is Yes, but the example given is also with FAILURE relationship (not RETRY relationship) 2. I can't find any session.commit in processor code. Is it OK? May it be, that the problem happens when session is not committed? Does NiFi any auto-commit then? 3. is there any design document for new processor regarding Penalize + RETRY/FAILURE relationship? Thanks. Michal
