What would be the right processor to use JsonTreeReader to get json
attributes as name/value pairs and to write them into the file?
Thanks again,
On Thu, Aug 30, 2018 at 2:09 PM Otto Fowler <ottobackwa...@gmail.com> wrote:
> What he said.
>
>
> On August 30, 2018 at 13:31:54, Joe Witt (joe.w...@gmail.com) wrote:
>
> Wow!
>
> My apologies for the really bad response I gave linking to the same
> article you mentioned. I should be more careful when
> reading/responding on the phone!
>
> Thanks
> On Thu, Aug 30, 2018 at 11:01 AM Matt Burgess <mattyb...@apache.org>
> wrote:
> >
> > V,
> >
> > Currently NiFi does not support specifying a schema in JSONSchema
> > format, you'll want to convert that to an Avro schema for use in
> > JsonTreeReader. I don't know JSONSchema that well so I'm not sure if
> > that "stats" schema is supposed to be included in the outgoing object.
> > I ran it through a utility-under-development [1] and got the following
> > Avro schema out:
> >
> >
> {"type":"record","name":"record0","fields":[{"name":"create_date","type":"long"},{"name":"id","type":"string"}]}
>
> >
> > This doesn't add the "stats" record and specifies "create_date" as a
> > long versus a BigInteger. I think you might want to use an Avro
> > logical type of "decimal" [2] depending on what the value is in the
> > actual JSON object:
> >
> > {"type":"record","name":"record0","fields":[
> > {"name":"create_date","type": {"type": "bytes","logicalType":
> > "decimal","precision": 12,"scale": 0}},
> > {"name":"id","type":"string"}
> > ]}
> >
> > If you have a stats object present, this might work:
> >
> > {"type":"record","name":"record0","fields":[
> > {"name": "stats", "type" :
> >
> {"type":"record","name":"statsRecord","fields":[{"name":"id","type":"string"},{"name":"bin_qualifier","type":"string"}]}},
>
> > {"name":"create_date","type": {"type": "bytes","logicalType":
> > "decimal","precision": 12,"scale": 0}},
> > {"name":"id","type":"string"}
> > ]}
> >
> > I didn't validate or try these so there may be typos or other
> > (hopefully minor) mistakes.
> >
> > Regards,
> > Matt
> >
> > [1] https://github.com/fge/json-schema-avro
> > [2] https://avro.apache.org/docs/1.8.2/spec.html#Decimal
> >
> > On Thu, Aug 30, 2018 at 9:54 AM l vic <lvic4...@gmail.com> wrote:
> > >
> > > I have json file for the schema that looks like the following:
> > >
> > > {
> > > "$schema": "http://json-schema.org/draft-04/schema#";,
> > > "definitions": {
> > > "stats": {
> > > "type": "object",
> > > "additionalProperties": false,
> > > "properties": {
> > > "id": {
> > > "type": "string"
> > > },
> > > "bin_qualifier": {
> > > "type": "string"
> > > }
> > > }
> > > }
> > > },
> > > "additionalProperties": false,
> > > "description": "attributes",
> > > "type": "object",
> > > "properties": {
> > > "id": {
> > > "type": "string",
> > > "required": true,
> > > },
> > > "create_date": {
> > > "type": "integer",
> > > "javaType": "java.math.BigInteger",
> > > "required": true
> > > }
> > > }
> > > }
> > >
> > >
> > > How can I add this schema for JsonTreeReader?
> > >
> > > On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <ottobackwa...@gmail.com>
> wrote:
> > >>
> > >> The record readers are services, that processors use.
> > >> When you use a *Record* processor, you will have to select a Reader
> and a Writer Service, or create one ( which you can do through the UI ).
> > >> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> > >>
> > >>
> > >> On August 30, 2018 at 08:48:08, l vic (lvic4...@gmail.com) wrote:
> > >>
> > >> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it
> in the list of available processors...
> > >> Thanks,
> > >> V
> > >>
> > >> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <
> sivaprasanna...@gmail.com> wrote:
> > >>>
> > >>> Hi. Just like CSVRecordReader, we have record reader service for
> JSON. It's called JsonTreeReader. You can use AvroSchemaRegistry and
> provide an Avro schema (usually generated through InferAvroSchema
> processor) for your JSON. Refer:
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
> > >>>
> > >>> -
> > >>> Sivaprasanna
> > >>>
> > >>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lvic4...@gmail.com> wrote:
> > >>>>
> > >>>> I need to save two different json messages according to json
> schemas available for each to different relational database tables.
> > >>>> I saw this blog:
> > >>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> > >>>> with example using CSVRecordReader for csv->json transformation.
> > >>>> but what would be RecordReader for schema-based transformation from
> json? Is this a valid approach, or what would be best approach to solve
> this problem?
> > >>>> I am using: nifi-1.7.1-RC1...
> > >>>> Thank you,
>
>
