GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/12271
[SPARK-14388][SQL][WIP] Implement CREATE TABLE ## What changes were proposed in this pull request? This patch implements the `CREATE TABLE` command using the `SessionCatalog`. Previously we handled only `CTAS` and `CREATE TABLE ... USING`. This requires us to refactor `CatalogTable` to accept various fields (e.g. bucket and skew columns) and pass them to Hive. ## How was this patch tested? Tests will come in a future commit. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewor14/spark create-table-ddl Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/12271.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12271 ---- commit 014c38e28e8f4545f926ef60ccb2ee4acae07b59 Author: Andrew Or <and...@databricks.com> Date: 2016-04-08T21:32:39Z Parse various parts of the CREATE TABLE command We need to reconcile the differences between what's added here in SparkSqlParser and HiveSqlParser. That will come in the next commit. This currently still fails tests, obviously because create table is not implemented yet! commit 15bb3b6c76e61d708538bee5d797981689ab6a8f Author: Andrew Or <and...@databricks.com> Date: 2016-04-01T21:20:37Z Refactor CatalogTable column semantics Before: CatalogTable has schema, partitionColumns and sortColumns. There are no constraints between the 3. However, Hive will complain if schema and partitionColumns overlap. After: CatalogTable has schema, partitionColumnNames, sortColumnNames, bucketColumnNames and skewColumnNames. All the columns must be a subset of schema. This means splitting up schema into (schema, partitionCols) before passing it to Hive. This allows us to store the columns more uniformly. Otherwise partition columns would be the odd one out. This commit also fixes "alter table bucketing", which was incorrectly using partition columns as bucket columns. commit b6b4d293c2efeb537110ef56fa9ffdcad90c9bb0 Author: Andrew Or <and...@databricks.com> Date: 2016-04-09T00:53:18Z Implement CREATE TABLE in Hive parser This involves reverting part of the changes in an earlier commit, where we tried to implement the parsing logic in the general SQL parser and introduced a bunch of case classes that we won't end up using. As of this commit the actual CREATE TABLE logic is not there yet. It will come in a future commit. commit 5e0fe03bfa655c6de854bc8adaa73186a17a0b0c Author: Andrew Or <and...@databricks.com> Date: 2016-04-09T06:52:20Z Implement it ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org