Re: Saving parquet table as uncompressed with write.mode("overwrite").

Checked default is gzip Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your

Re: Saving parquet table as uncompressed with write.mode("overwrite").

thanks Ted that was it :) scala> val c = sqlContext.setConf("spark.sql.parquet.compression.codec", "uncompressed") c: Unit = () scala> val s4 = s.write.mode("overwrite").parquet("/user/hduser/sales4") s4: Unit = () Before -rw-r--r-- 2 hduser supergroup 17487 2016-07-03 22:28

Re: Saving parquet table as uncompressed with write.mode("overwrite").

Have you tried the following (note the extraneous dot in your config name) ? val c = sqlContext.setConf("spark.sql.parquet.compression.codec", "none") Also, parquet() has compression parameter which defaults to None FYI On Sun, Jul 3, 2016 at 2:42 PM, Mich Talebzadeh

Saving parquet table as uncompressed with write.mode("overwrite").

Hi, I simply read a Parquet table scala> val s = sqlContext.read.parquet("oraclehadoop.sales2") s: org.apache.spark.sql.DataFrame = [prod_id: bigint, cust_id: bigint, time_id: timestamp, channel_id: bigint, promo_id: bigint, quantity_sold: decimal(10,0), amount_sold: decimal(10,0)] Now all I