I think what he is saying is that the csv files may not always have all the columns, in which case they won’t parse, which is before stellar can do anything.
On November 16, 2019 at 11:30:25, Simon Elliston Ball ( [email protected]) wrote: A better way of doing this would be to use the fieldTransformation setting and the REMOVE method to get rid of the extraneous fields. Docs are included at https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html# That way you don’t need a separate preprocessing step. Simon On Sat, 16 Nov 2019 at 16:09, Hema malini <[email protected]> wrote: > Thanks ..will do preprocessing of data.. > > On Sat, 16 Nov, 2019, 9:25 PM Otto Fowler, <[email protected]> > wrote: > >> No, there is no way to do this currently. >> >> The parser parses the line into and array of strings that must match the >> size of the columns. >> >> The underlying opencsv parser does not support this either. You may have >> to do some normalization work on your data if you need to account for this. >> >> >> >> On November 16, 2019 at 08:49:36, Hema malini ([email protected]) >> wrote: >> >> Hi all, >> >> Is there any way to mark some columns as optional in column mapping in >> CSV parser. >> >> Thanks and Regards, >> Hema >> >> -- -- simon elliston ball @sireb
