Thanks a lot for your timely response Simon and Otto . On Sun, 17 Nov, 2019, 9:20 PM Otto Fowler, <ottobackwa...@gmail.com> wrote:
> I think what he is saying is that the csv files may not always have all > the columns, in which case they won’t parse, which is before stellar can do > anything. > > > > > On November 16, 2019 at 11:30:25, Simon Elliston Ball ( > si...@simonellistonball.com) wrote: > > A better way of doing this would be to use the fieldTransformation setting > and the REMOVE method to get rid of the extraneous fields. Docs are > included at > > https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html# > > That way you don’t need a separate preprocessing step. > > Simon > > On Sat, 16 Nov 2019 at 16:09, Hema malini <nhemamalin...@gmail.com> wrote: > >> Thanks ..will do preprocessing of data.. >> >> On Sat, 16 Nov, 2019, 9:25 PM Otto Fowler, <ottobackwa...@gmail.com> >> wrote: >> >>> No, there is no way to do this currently. >>> >>> The parser parses the line into and array of strings that must match the >>> size of the columns. >>> >>> The underlying opencsv parser does not support this either. You may >>> have to do some normalization work on your data if you need to account for >>> this. >>> >>> >>> >>> On November 16, 2019 at 08:49:36, Hema malini (nhemamalin...@gmail.com) >>> wrote: >>> >>> Hi all, >>> >>> Is there any way to mark some columns as optional in column mapping in >>> CSV parser. >>> >>> Thanks and Regards, >>> Hema >>> >>> -- > -- > simon elliston ball > @sireb > >
