[ https://issues.apache.org/jira/browse/SPARK-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-17166. ----------------------------------- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.4.0 It is decided to support Spark-related Parquet/ORC properties and implemented in Spark 2.4.0. The other properties which are unknown to Spark will be ignored intentionally. > CTAS lost table properties after conversion to data source tables. > ------------------------------------------------------------------ > > Key: SPARK-17166 > URL: https://issues.apache.org/jira/browse/SPARK-17166 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.0.0 > Reporter: Xiao Li > Assignee: Dongjoon Hyun > Priority: Major > Fix For: 2.4.0 > > > CTAS lost table properties after conversion to data source tables. For > example, > {noformat} > CREATE TABLE t TBLPROPERTIES('prop1' = 'c', 'prop2' = 'd') AS SELECT 1 as a, > 1 as b > {noformat} > The output of `DESC FORMATTED t` does not have the related properties. > {noformat} > |Table Parameters: | > | | > | rawDataSize |-1 > | | > | numFiles |1 > | | > | transient_lastDdlTime |1471670983 > | | > | totalSize |496 > | | > | spark.sql.sources.provider|parquet > | | > | EXTERNAL |FALSE > | | > | COLUMN_STATS_ACCURATE |false > | | > | numRows |-1 > | | > | | > | | > |# Storage Information | > | | > |SerDe Library: > |org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe > | | > |InputFormat: > |org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat > | | > |OutputFormat: > |org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat > | | > |Compressed: |No > | | > |Storage Desc Parameters: | > | | > | serialization.format |1 > | | > | path > |file:/private/var/folders/4b/sgmfldk15js406vk7lw5llzw0000gn/T/warehouse-f3aa2927-6464-4a35-a715-1300dde6c614/t| > | > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org