[ https://issues.apache.org/jira/browse/SPARK-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust updated SPARK-18543: ------------------------------------- Target Version/s: 2.3.0 (was: 2.2.0) > SaveAsTable(CTAS) using overwrite could change table definition > --------------------------------------------------------------- > > Key: SPARK-18543 > URL: https://issues.apache.org/jira/browse/SPARK-18543 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.6.3, 2.0.2 > Reporter: Xiao Li > Assignee: Xiao Li > > When the mode is OVERWRITE, we drop the Hive serde tables and create a data > source table. This is not right. > {code} > val tableName = "tab1" > withTable(tableName) { > sql(s"CREATE TABLE $tableName STORED AS SEQUENCEFILE AS SELECT 1 AS > key, 'abc' AS value") > val df = sql(s"SELECT key, value FROM $tableName") > df.write.mode(SaveMode.Overwrite).saveAsTable(tableName) > val tableMeta = > spark.sessionState.catalog.getTableMetadata(TableIdentifier(tableName)) > assert(tableMeta.provider == > Some(spark.sessionState.conf.defaultDataSourceName)) > } > {code} > Based on the definition of OVERWRITE, no change should be made on the table > definition. When recreate the table, we need to create a Hive serde table. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org