On Mon, 7 Mar 2022 11:52:02 -0800 HK Verma <hkve...@gmail.com> wrote:
> Thanks Antoine. Yes I have newlines_in_values set to false. Other configs > also look ok. > However I do have rows with less number of columns than the specified > numbers in convert options in column types. I have my own > invalid_row_handler where I currently skip these rows. > It looks like the parser is doing a quick pass and splitting the blocks > using the new line separator and parsing them in parallel. This seems > similar to spark csv multiline option. To avoid this case, I did set > newlines_in_values to false and also checked if newlines are not there in > any of the fields. No, you should check `newlines_in_values` to true and see if the error goes away. Regards Antoine.
