Anders created NIFI-10066:
-----------------------------
Summary: Option in ValidateRecord to only send failed record to
failure relationship
Key: NIFI-10066
URL: https://issues.apache.org/jira/browse/NIFI-10066
Project: Apache NiFi
Issue Type: Improvement
Reporter: Anders
Currently, the ValidateRecord processor treats invalid and failure differently.
With invalid records (records passing parsing but not validaiton), they are
sent to the invalid relationship, while valid records in the flowfile are sent
to the valid relationship.
Whenever the processor encounters a junk records (random text, invalid synax,
incomplete json, etc.), the _entire_ flowfile is sent to the failure
relationship, not just the failed record.
I am sure there are use cases where this makes sense, but I currently have a
use-case where I get large flowfiles with a large amount of json objects. Every
now and then a junk (non-json) line gets into the (flow)files, and I want to
filter them out.
It would be very usefull if ValidateRecord could _only_ send those junk records
to the failure relationship, and not the entire flowfile. Currently, I have to
split everything to the failure relationship in single line flowfiles, and run
them by ValidateRecord again to identify only the junk records, and send the
remaining valid records onward in my flow.
This results in needlessly complicated logic in the flow.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
