[ https://issues.apache.org/jira/browse/NIFI-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244208#comment-17244208 ]
ASF subversion and git services commented on NIFI-8068: ------------------------------------------------------- Commit 37c2284d04ccf7ceb0d8ede62dafe6dce5ab835e in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=37c2284 ] NIFI-8068: Ensure that when we determine the best of multiple possible types in a UNION that we handle Arrays of Records properly. Also refactored code to be a bit cleaner by extracting blocks of it into appropriately named methods Signed-off-by: Pierre Villard <pierre.villard...@gmail.com> This closes #4706. > JsonReader fails when reading a Record if the schema declares data type as a > union of String or Array<Record> > ------------------------------------------------------------------------------------------------------------- > > Key: NIFI-8068 > URL: https://issues.apache.org/jira/browse/NIFI-8068 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Reporter: Mark Payne > Assignee: Mark Payne > Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > If I attempt to read the following JSON with an inferred schema: > {code:java} > { > "fields": [ > {"type": "string"}, > {"type": [{"type": "string"}]} > ] > }{code} > The Record Reader fails with the following stack trace: > {code:java} > 2020-12-03 12:07:34,310 ERROR [Timer-Driven Process Thread-6] > o.a.n.processors.standard.ConvertRecord > ConvertRecord[id=2991bff1-0176-1000-7771-3866d63fc7d3] Failed to process > StandardFlowFileRecord[uuid=86483188-4d5e-40a9-bf2a-f570e6656f0f,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1607015254266-1, container=default, > section=1], offset=0, > length=83],offset=0,name=86483188-4d5e-40a9-bf2a-f570e6656f0f,size=83]; will > route to failure: org.apache.nifi.processor.exception.ProcessException: Could > not parse incoming data > org.apache.nifi.processor.exception.ProcessException: Could not parse > incoming data > at > org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:171) > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2986) > at > org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173) > at > org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.nifi.serialization.MalformedRecordException: > Successfully parsed a JSON object from input but failed to convert into a > Record object with the given schema > at > org.apache.nifi.json.AbstractJsonRowRecordReader.nextRecord(AbstractJsonRowRecordReader.java:124) > at > org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240) > at com.sun.proxy.$Proxy138.nextRecord(Unknown Source) > at > org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:131) > ... 14 common frames omitted > Caused by: java.lang.ClassCastException: > org.apache.nifi.serialization.record.MapRecord cannot be cast to > java.lang.Byte > at > org.apache.nifi.serialization.record.util.DataTypeUtils.toString(DataTypeUtils.java:925) > at > org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:203) > at > org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:148) > at > org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:224) > at > org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:148) > at > org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:144) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertField(JsonTreeRowRecordReader.java:215) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:124) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertField(JsonTreeRowRecordReader.java:209) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertField(JsonTreeRowRecordReader.java:184) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:124) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:70) > at > org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:61) > at > org.apache.nifi.json.AbstractJsonRowRecordReader.nextRecord(AbstractJsonRowRecordReader.java:119) > ... 24 common frames omitted {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)