It will 'auto-detect' the compression codec by the file extension and then will decompress and read it correctly.
Thanks! 2016-06-16 20:27 GMT+09:00 Vamsi Krishna <vamsi.attl...@gmail.com>: > Hi, > > I'm using Spark 1.4.1 (HDP 2.3.2). > As per the spark-csv documentation ( > https://github.com/databricks/spark-csv), I see that we can write to a > csv file in compressed form using the 'codec' option. > But, didn't see the support for 'codec' option to read a csv file. > > Is there a way to read a compressed (gzip) file using spark-csv? > > Thanks, > Vamsi Attluri > -- > Vamsi Attluri >
