Hello Matt, I tried to use the UpdateRecord processor, but it seems that it doesn't work with map types. Suppose the name of the map is "primitives" and has a key "a". If I define the property /a = /primitives/a , the value of "a" is always null
Regards, Chris On Wed, Oct 11, 2017 at 6:18 PM, Matt Burgess <[email protected]> wrote: > If you know the input and output schemas, you should be able to use > UpdateRecord for this. You would have a user-defined property for each > output field (including flattened field names), whose value would be > the RecordPath to the field in the map. I believe for any fields that > are in the input schema but not the output schema, they are ignored > (but don't quote me on this, give it a try ;) so you could effectively > "convert" the record by updating outgoing fields with the values from > the nested fields in the input record. > > Regards, > Matt > > On Wed, Oct 11, 2017 at 12:10 PM, Bryan Bende <[email protected]> wrote: > > Casting string to integer wouldn't be a problem, but flattening a map > would be. > > > > I think there was a previous thread where a FlattenRecord processor > > was discussed, and I believe there was agreement that it would be > > useful, but it just hasn't been developed yet. > > > > If you had FlattenRecord then you would probably bring the data in > > from ConsumeKafkaRecord keeping the original schema (using a writer > > with "inherit schema from reader") and then later in the flow do a > > FlattenRecord. > > > > As a current option, you might be able to do the flattening yourself > > with an ExecuteScript processor, or maybe a scripted record writer > > that did the flattening on the way out. > > > > > > On Wed, Oct 11, 2017 at 11:22 AM, Chris Herssens > > <[email protected]> wrote: > >> Hello Bryan, > >> > >> In my case the schemas are not compatible, In Schema A I have a map > type and > >> I want to flatten this map. Also I want to cast string to integer. > >> > >> Chris > >> > >> On Wed, Oct 11, 2017 at 5:03 PM, Bryan Bende <[email protected]> wrote: > >>> > >>> Chris, > >>> > >>> Any processor that uses a record reader and record writer can > >>> inherently do schema conversion by using schema A for the reader, and > >>> schema B for the writer, assuming the schemas are compatible. > >>> > >>> Compatible in this sense would mean one of the following... > >>> > >>> - Schema B has the same field names as schema A, but with some of the > >>> filed types being different > >>> - Schema B has a subset of the fields in schema A, possibly some > >>> changing type as above > >>> - Schema B has additional fields and they have default values since > >>> they won't exist in the records coming from schema A > >>> > >>> If you have Avro data in Kafka that already has the schema embedded in > >>> it, then you can use ConsumeKafkaRecord with an AvroReader and set the > >>> Schema Access Strategy to "Embedded Avro", and then use a > >>> AvroRecordSetWriter and set the Schema Access Strategy to one of the > >>> other options like Schema Name (which needs a schema registry) or > >>> Schema Text which allows you to enter in a full schema. > >>> > >>> You could also do the same thing anywhere else in your flow using > >>> ConvertRecord. > >>> > >>> Thanks, > >>> > >>> Bryan > >>> > >>> > >>> On Wed, Oct 11, 2017 at 7:55 AM, Chris Herssens > >>> <[email protected]> wrote: > >>> > Hello All, > >>> > > >>> > I would like to convert an avro schema to another avro schema. > Since > >>> > Nifi > >>> > reads the avro data from kafka, I can't use the ConvertAvroSchema > >>> > processor. > >>> > Which processor can I use ? > >>> > Is it possible to use ConsumeKafkaRecord processor for that ? If yes > how > >>> > do > >>> > we specify the dynamic properties ? If possible can you give me an > >>> > example > >>> > ? > >>> > > >>> > Regards, > >>> > > >>> > Chris > >> > >> >
