[ https://issues.apache.org/jira/browse/NIFI-12745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajmund Takacs updated NIFI-12745: ---------------------------------- Attachment: ValidateRecord.json > AvroReader silently drops record if it's malformed > -------------------------------------------------- > > Key: NIFI-12745 > URL: https://issues.apache.org/jira/browse/NIFI-12745 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework > Affects Versions: 2.0.0-M1, 1.18.0, 1.19.0, 1.20.0, 1.19.1, 1.21.0, > 1.22.0, 1.23.0, 1.24.0, 1.23.1, 1.23.2, 1.25.0, 2.0.0-M2 > Reporter: Rajmund Takacs > Assignee: Rajmund Takacs > Priority: Major > Attachments: ValidateRecord.json > > Time Spent: 10m > Remaining Estimate: 0h > > See the attached example flow. It reproduces the issue very reliably. > {{GenerateFlowFile}} is set to generate the following Json: > {code:json} > [{ > "field_1" : 123456789, > "field_2" : "44", > "field_3" : 5 > }] > {code} > This input is converted to Avro format, using the {{ConvertRecord}} > processor. The 'Schema Write Strategy' of {{AvroRecordSetWriter}} is set to > anything different than 'Embed Avro Schema'. > Then, the resulting FF is routed to a processor that uses an {{AvroReader}} > to work on the records. The reader is set to use a predefined, fixed schema, > which does not match with the input avro file, contains at least an extra > field. It does not matter if that field has a default value or not. > {code:json} > { > "type":"record", > "name":"message_name", > "namespace":"message_namespace", > "fields":[ > { > "name":"field_1", > "type":["long"] > }, > { > "name":"field_2", > "type":["string"] > }, > { > "name":"field_3", > "type":["int"] > }, > { > "name":"extra_field", > "type":["string"], > "default":"empty" > } > ] > } > {code} > When this processor consumes the input, the reader silently drops the record, > without even making an error log message. At the processor level, this is > equivalent to having no records to process, so nothing happens. The user > won't notice that there is a misconfiguration somewhere until they start > noticing the missing the flow files. > The expected behavior from the processors would be to route the malformed > input FF to their failure relationship, and shout an error on its bulletin. -- This message was sent by Atlassian Jira (v8.20.10#820010)