Possibly also related to NIFI-8135 [1] (fix to be included in 1.24.0 when released) and/or NIFI-8134 [2], with an open PR [3] that’s yet to be reviewed.
[1] https://issues.apache.org/jira/browse/NIFI-8135 [2] https://issues.apache.org/jira/browse/NIFI-8134 [3] https://github.com/apache/nifi/pull/7745 Cheers, --- Chris Sampson IT Consultant chris.samp...@naimuri.com > On 22 Nov 2023, at 09:42, Lehel Boér <lehe...@hotmail.com> wrote: > > Hi Aaron, > > You need to define the schema for the "data" field as a nested record. > > { > "name": "data", > "type": { > "type": "record", > "name": "nestedData", > "fields": [ > {"name": "message_type", "type": "string"}, > {"name": "message_time", "type": "string"}, > {"name": "file_ingest_time", "type": "string"} > ] > } > } > > Best Regards, > Lehel > From: Aaron Rich <aaron.r...@gmail.com> > Sent: Wednesday, November 22, 2023 7:30 > To: d...@nifi.apache.org <d...@nifi.apache.org>; users@nifi.apache.org > <users@nifi.apache.org> > Subject: RecordReader/Writer with nested json resulting in MapRecord[] > statement > > Hello, > > I'm trying to use the record writer/writers to take a json defined event and > write it to postgres database. The event follows the CloudEvent definition > and has a data element that is JSON structured: > > { > "data" : { > "message_type" : "V5", > "message_time" : "1614597071000", > "file_ingest_time": "1682018983320" > }, > "spec_version" : "1.0.2", > "subject" : "1969", > "data_schema" : "URI://something", > "id" : "80e12364-831e-446a-a260-86bccc469f25", > "source" : "service", > "time" : "2023-11-22T00:03:03Z", > "type" : "V5", > "data_content_type" : "application/json" > } > > I'm trying to write into a postgres db table that has each field as a column > and data as a JSON type column. > > When I try to write the the database with PutDatabaseRecord, I'm getting > MapRecord sting that is trying to be written into the data column: > MapRecord[{message_type=V5, > message_time=1614597071000,file_ingest_time=1682018983320}] > > I _think_ the issue is with the schema I'm defining: > { > "type": "record", > > "name": "eventSchema0", > > "fields" : [ > {"name":"data", "type": "string"}, > {"name":"spec_version", "type": "string"}, > {"name":"subject", "type": "string"}, > {"name":"data_schema", "type": "string"}, > {"name":"id", "type": "string"}, > {"name":"source", "type": "string"}, > {"name":"time", "type": "string"}, > {"name":"type", "type": "string"}, > {"name":"data_content_type", "type": "string"} > ] > > } > > What do I need to change so the JSON structure in the "data" field can be > properly written using a record based processor? > > Thanks for the help. > > -Aaron
