Thank you for your suggestionsâș
It worked
On 10-Jul-2017 7:26 PM, "Bryan Bende" <bbe...@gmail.com> wrote:
> I think FreeFormTextRecordSetWriter can only access fields from the
> read schema, so you are correct that this would be a problem since
> "Sex" wasn't in the read schema.
>
> You could change your CSVReader to have 'Schema Access Strategy" ->
> "Use Schema Name" and then create an AvroSchemaRegistry and define a
> schema with the "Sex" field like:
>
> {
> "name": "custom",
> "namespace": "nifi",
> "type": "record",
> "fields": [
> { "name": "No", "type": "int" },
> { "name": "Name", "type": "string" },
> { "name": "ID", "type": "int" },
> { "name": "Age", "type": "int" },
> { "name": "Sex", "type": ["string", "null"] },
> ]
> }
>
> Make sure your flow file has an attribute "schema.name" with the name
> that you used when you added the above schema to the
> AvroSchemaRegistry.
>
>
>
> On Mon, Jul 10, 2017 at 1:16 AM, prabhu Mahendran
> <prabhuu161...@gmail.com> wrote:
> > Hi bryan,
> >
> > Thanks for your suggestion.
> >
> > i have followed your steps and am have an one doubt regarding your schema
> > creation for "Sex" column.
> >
> > i haven't using schema in created flow.
> >
> > You have said that flow have a field called "Sex" in the schema being
> used
> > by the record writer.i'm FreeFormSetWriter only not "CSVRecordWriter"
> >
> > In that CSVReader i have configured "Schema Access Strategy"-->"Use
> String
> > fields from Header".
> >
> > FreeFormSetWriter,i have specified the ${ID},${Name},${Sex} only.
> >
> > i wants you to know when i have to create schema for "Sex".
> >
> > Please stop me if am understands anything wrong.
> >
> > Can you please guide me to create schema for my requirement?
> >
> > Thanks,
> > prabhu
> >
> > On Fri, Jul 7, 2017 at 5:05 PM, Bryan Bende <bbe...@gmail.com> wrote:
> >>
> >> Hi Prabhu,
> >>
> >> The SimpleCsvFileLookupService is meant to look up one value and add
> >> it back to the records.
> >>
> >> So lets say you want to lookup the gender and add it to the original
> >> records...
> >>
> >> You would configure SimpleCsvFileLookupService with the following:
> >>
> >> - Lookup Key Column = ID
> >> - Lookup Key Value = Sex
> >>
> >> When the service starts it will then make a map of ID to Sex so you
> would
> >> have:
> >>
> >> 2201 -> Male
> >> 3300 -> Female
> >>
> >> Now in LookupRecord you would add a user-defined property of "key" =
> >> "ID" since ID is the column from the incoming records that would to
> >> use as the key into the above map.
> >>
> >> Then "Result Record Path" should be the field in the records where you
> >> want the result of the look up go to, so you would want something like
> >> "/Sex".
> >>
> >> You'll also need to have a field called "Sex" in the schema being used
> >> by the record writer. You could make one schema that has a nullable
> >> Sex field and have the CsvReader and FreeFormTextWriter both reference
> >> that schema, or you could let the CsvReader infer the schema from the
> >> fields (it won't have a sex field) and then use a different schema for
> >> the writer.
> >>
> >> -Bryan
> >>
> >>
> >> On Fri, Jul 7, 2017 at 6:28 AM, prabhu Mahendran
> >> <prabhuu161...@gmail.com> wrote:
> >> > I tried to join two csv file based on id with respect to the below
> >> > reference.
> >> >
> >> >
> >> > How to join two CSVs with Apache Nifi
> >> >
> >> >
> >> > i'm using NiFi-1.3.0
> >> >
> >> >
> >> > Now i have two csv files.
> >> >
> >> >
> >> > 1.custom.csv
> >> >
> >> >
> >> > No,Name,ID,Age
> >> >
> >> > 1,Hik,2201,33
> >> >
> >> > 2,Kio,3300,22
> >> >
> >> >
> >> > 2.gender.csv
> >> >
> >> >
> >> > ID,Name,Sex
> >> >
> >> > 2201,Hik,Male
> >> >
> >> > 3300,Kio,Female
> >> >
> >> > I try to combine those tables with "ID" like following endresult.
> >> >
> >> >
> >> > No,Name,Sex,ID,Age
> >> >
> >> > 1,Hik,Male,2201,33
> >> >
> >> > 2,Kio,Female,3300,22
> >> >
> >> > I have using following processor structure.
> >> >
> >> >
> >> > GetFile-SplitText-ExtractText-LookUpRecord-PutFile
> >> >
> >> > In that lookup record i have configured
> >> >
> >> >
> >> > RecordReader = "CSVReader"
> >> >
> >> > RecordWriter="FreeFormTextRecordSetWriter"
> >> >
> >> > LookUpService="SimpleCSVFileLookUpService"
> >> >
> >> >
> >> > ResultRecordPath-->/Key
> >> >
> >> > key-->/ID
> >> >
> >> > In that LookUpService i have given path of the "gender.csv" and setted
> >> > LookUpKeyColumn and LookUpValueColumn to be "ID".
> >> >
> >> >
> >> > In that FreeFormTextRecordSetWriter i have given text
> >> > value"${No},${Name},${ID},${Age},${Sex}"
> >> >
> >> >
> >> > It yields below result only.
> >> >
> >> >
> >> > No,Name,Sex,ID,Age,
> >> >
> >> > 1,Hik,Male,2201,33,
> >> >
> >> > 2,Kio,Female,3300,22,
> >> >
> >> > It doesn't have "sex" column.
> >> >
> >> >
> >> > I think i could not configured correctly.
> >> >
> >> >
> >> > i don't know how to use ResultRecordPath & one dynamic attribute(Key)
> >> > specification in LookUpRecord?
> >> >
> >> >
> >> > Can anyone guide me to solve my issue?
> >
> >
>
