[
https://issues.apache.org/jira/browse/SPARK-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated SPARK-14130:
-----------------------------
Description:
For alter column command, we have the following tokens.
TOK_ALTERTABLE_RENAMECOL
TOK_ALTERTABLE_ADDCOLS
TOK_ALTERTABLE_REPLACECOLS
For data source tables, we should throw exceptions. For Hive tables, we should
support them. *For Hive tables, we should check Hive's behavior to see if there
is any file format that does not any of above command*.
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
is a good reference for Hive's behavior.
Also, for a Hive table stored in a format, we need to make sure that even if
Spark can read this tables after an alter column operation. If we cannot read
the table, even Hive allows the alter column operation, we should still throw
an exception. For example, if renaming a column of a Hive parquet table causes
the renamed column inaccessible (we cannot read values), we should not allow
this renaming operation.
was:
For alter column command, we have the following tokens.
TOK_ALTERTABLE_RENAMECOL
TOK_ALTERTABLE_ADDCOLS
TOK_ALTERTABLE_REPLACECOLS
For data source tables, we should throw exceptions. For Hive tables, we should
support them. *For Hive tables, we should check Hive's behavior to see if there
is any file format that does not any of above command*.
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
is a good reference for Hive's behavior.
> [Table related commands] Alter column
> -------------------------------------
>
> Key: SPARK-14130
> URL: https://issues.apache.org/jira/browse/SPARK-14130
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Yin Huai
>
> For alter column command, we have the following tokens.
> TOK_ALTERTABLE_RENAMECOL
> TOK_ALTERTABLE_ADDCOLS
> TOK_ALTERTABLE_REPLACECOLS
> For data source tables, we should throw exceptions. For Hive tables, we
> should support them. *For Hive tables, we should check Hive's behavior to see
> if there is any file format that does not any of above command*.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
> is a good reference for Hive's behavior.
> Also, for a Hive table stored in a format, we need to make sure that even if
> Spark can read this tables after an alter column operation. If we cannot read
> the table, even Hive allows the alter column operation, we should still throw
> an exception. For example, if renaming a column of a Hive parquet table
> causes the renamed column inaccessible (we cannot read values), we should not
> allow this renaming operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
