Jinhua Fu created SPARK-21786:
---------------------------------
Summary: The 'spark.sql.parquet.compression.codec' configuration
doesn't take effect on tables with partition field(s)
Key: SPARK-21786
URL: https://issues.apache.org/jira/browse/SPARK-21786
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.2.0
Reporter: Jinhua Fu
For tables created like below, 'spark.sql.parquet.compression.codec' doesn't
take any effect when insert data. And because the default compression codec is
'uncompressed', if I want to change the compression codec, I have to change it
by 'set parquet.compression=xxxx'.
Contrast，tables without any partition field will work normal with
'spark.sql.parquet.compression.codec',and the default compression codec is
'snappy', but it seems 'parquet.compression' no longer in effect.
Should we use the ‘spark.sql.parquet.compression.codec’ configuration uniformly？
----------------------------------------------------------------------------------------------------
CREATE TABLE Test_Parquet(provincecode int, citycode int, districtcode int)
PARTITIONED BY (p_provincecode int)
STORED AS PARQUET;
INSERT OVERWRITE TABLE Test_Parquet select * from TableB;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
