Re: CsvInputFormat - read header line first

2018-10-31 Thread madan
Hi Ken, Yep correct. Thank you. On Wed, Oct 31, 2018 at 7:24 PM Ken Krugler wrote: > Hi Madan, > > If your source has a parallelism > 1, then when the CSV file is split, > only one of the operators will get the split with the header row. > > So in that case, how would you communicate the

Re: CsvInputFormat - read header line first

2018-10-31 Thread Ken Krugler
Hi Madan, If your source has a parallelism > 1, then when the CSV file is split, only one of the operators will get the split with the header row. So in that case, how would you communicate the column name->index information to the other operators? If you force a parallelism of 1 for the

CsvInputFormat - read header line first

2018-10-30 Thread madan
Hi, When we are splitting a csv file into multiple parts we are not sure which part is read first. Is there any way to make sure first part with header is read first ? I need to read header line first to store column name vs index and use this index for processing next records. I could read