Re: Read CSV file and and create customized field

[Chesnay Schepler]

You should add an extra map function.

On 16/01/2020 17:10, Soheil Pourbafrani wrote:

Hi friends,

I'm going to read a CSV file that has 3 columns. I want the final loaded datatype to have other columns inferred by that 3 columns. For example, I would split the first column of the CSV file and create 3 new columns. The problem is I did not find a straightforward approach for that. Here is what I have so far:

env.readCsvFile("pathToCsv")
         .fieldDelimiter(",")
         .ignoreFirstLine()
         .ignoreInvalidLines()
         .type(String.class, String.class, String.class)
         .print();

So is there any way to show the readCSVFile how to split CSV records or I should add an extra map function after loading CSV to create my desired schema?

Thanks