Re: Fwd: Spark 2.0 Shell -csv package weirdness

Have u tried df.saveAsParquetFIle? I think that method is on df Api Hth Marco On 19 Mar 2016 7:18 pm, "Vincent Ohprecio" wrote: > > For some reason writing data from Spark shell to csv using the `csv > package` takes almost an hour to dump to disk. Am I going crazy or did I

Fwd: Spark 2.0 Shell -csv package weirdness

For some reason writing data from Spark shell to csv using the `csv package` takes almost an hour to dump to disk. Am I going crazy or did I do this wrong? I tried writing to parquet first and its fast as normal. On my Macbook Pro 16g - 2.2 GHz Intel Core i7 -1TB the machine CPU's goes crazy and