[ https://issues.apache.org/jira/browse/NIFI-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Payne updated NIFI-10660: ------------------------------ Status: Patch Available (was: Open) > AvroReader issue after upgrading to Nifi 1.18.0 > ----------------------------------------------- > > Key: NIFI-10660 > URL: https://issues.apache.org/jira/browse/NIFI-10660 > Project: Apache NiFi > Issue Type: Bug > Affects Versions: 1.18.0 > Reporter: Sandro Berger > Assignee: Mark Payne > Priority: Critical > Fix For: 1.19.0 > > Time Spent: 10m > Remaining Estimate: 0h > > After upgrading from 1.15.3 to 1.18.0 we saw some issue regarding AvroReader > when the following conditions are met: > # The AVRO schema in the flowFile uses the "aliases" attribute > # There is a record in the flowFile where a field with set alias has a value > of "null" > The use of aliases and null values used to work for us before upgrading Nifi > without issues > Example schema: > {code:java} > { > "type": "record", > "name": "test", > "fields": [ > { "name" : "id", "type" : "int", "aliases" : ["F1"] }, > { "name" : "field1", "type" : [ "string", "null" ], "aliases" : ["F2"] > }, > { "name" : "field2", "type" : [ "string", "null" ], "aliases" : ["F3"] } > ] > } > {code} > Example data: > {code:java} > [ { > "id" : 1, > "field1" : "field1content", > "field2" : "field2content" > }, { > "id" : 2, > "field1" : null, > "field2" : "field2content" > } ]{code} > Note: If we remove the "aliases" from the schema or make sure that there is > never a field with a value of null, we don't have any errors in Nifi 1.18.0. > > ERROR log: > {code:java} > 2022-10-14 16:30:50,617 ERROR [Timer-Driven Process Thread-6] > o.a.n.processors.standard.ConvertRecord > ConvertRecord[id=c7ee1f0f-10ac-116e-9e4c-300530853ab8] Failed to process > StandardFlow > FileRecord[uuid=1d201fdb-e507-4d51-87c2-fbc895a450f3,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1665753427357-4670, > container=default, section=574], offset=18384, > length=324],offset=0,name=1d201fdb-e507-4d51-87c2-fbc895a450f3,size=324]; > will route to failure > org.apache.nifi.processor.exception.ProcessException: Could not parse > incoming data > at > org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:171) > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3434) > at > org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1354) > at > org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246) > at > org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > Caused by: org.apache.nifi.serialization.MalformedRecordException: Error > while getting next record > at > org.apache.nifi.avro.AvroRecordReader.nextRecord(AvroRecordReader.java:50) > at > org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50) > at jdk.internal.reflect.GeneratedMethodAccessor150.invoke(Unknown > Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240) > at com.sun.proxy.$Proxy277.nextRecord(Unknown Source) > at > org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:157) > ... 13 common frames omitted > Caused by: org.apache.avro.AvroRuntimeException: Not a valid schema field: F2 > at > org.apache.avro.generic.GenericData$Record.get(GenericData.java:268) > at > org.apache.nifi.avro.AvroTypeUtil.convertAvroRecordToMap(AvroTypeUtil.java:916) > at > org.apache.nifi.avro.AvroTypeUtil.convertAvroRecordToMap(AvroTypeUtil.java:905) > at > org.apache.nifi.avro.AvroRecordReader.nextRecord(AvroRecordReader.java:43) > ... 22 common frames omitted {code} > The error mentions the alias of the field containing the null value. > Could be related to the upgrade of avro introduced with Nifi 1.18.0: > https://issues.apache.org/jira/browse/NIFI-10311 -- This message was sent by Atlassian Jira (v8.20.10#820010)