Re: Spark CSV Quote only NOT NULL

Glad to help! On Sat, Jul 13, 2019 at 12:17 PM Gourav Sengupta wrote: > Hi Swetha, > I always look into the source code a lot but it never occured to me to > look into the test suite, thank a ton for the tip. Does definitely give > quite a few ideas - thanks a ton. > > Thanks and Regards, >

Re: Spark CSV Quote only NOT NULL

Hi Swetha, I always look into the source code a lot but it never occured to me to look into the test suite, thank a ton for the tip. Does definitely give quite a few ideas - thanks a ton. Thanks and Regards, Gourav On Fri, Jul 12, 2019 at 6:51 AM Swetha Ramaiah wrote: > Hi Anil > > That was

write csv does not handle \r correctly

hi spark 2.4.1 The csv writer does not quote string columns when they contain the \r carriage return character. It works as expected for both \n and \r\n \r is considered as a newline by many parsers, and spark should consider it as a newline marker. thanks -- nicolas

timestamp column orc problem with hive

Hi spark 2.4.1 hive 1.2.2 The orc files saved as tables from spark are not working correctly with hive. A "timestampCol is not null" does not work as expected. The parquet format works as expected for the same input. Is this a known issue ? thanks -- nicolas