Wenzhe Zhou has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/19383 )
Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu ...................................................................... WIP IMPALA-11809: Support non unique primary key for Kudu Kudu engine recently adds support for non unique primary key. Kudu server automatically adds an auto-incrementing column named 'auto_increment_id' in a Kudu table which has non unique primary key. The non unique primary key and 'auto_increment_id' form the effective unique composite primary key. This patch integrated new version of Kudu which support non unique primary key, added syntactic support for creating table with non unique primary key. Example: CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING) PARTITION BY HASH (i) partitions 3 STORED as KUDU; CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i)) PARTITION BY HASH (i) partitions 3 STORED as KUDU; CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id) PARTITION BY HASH (id) partitions 3 STORED as KUDU AS SELECT id, string_col FROM functional.alltypes WHERE id = 10; Kudu assign values for auto-incrementing column automatically when inserting rows so insertion statements don't need to specify values for auto-incrementing column. SELECT statement does not show auto-incrementing column unless the column is explicitly specified in select list. UPSERT operation is not supported now for Kudu table with auto incrementing column due to limitation in Kudu engine. When creating a Kudu table, specifying PRIMARY KEY is optional. If there is no primary key attribute specified, the partition key columes will be promoted as non unique primary key if those columns are the beginning columns of the table. New column "key_unique" is added to the output of 'describe' table command for Kudu table. Testing: - Integrated new version of Kudu built on local machine, ran manual test in impala-shell with queries to create tables with non unique primary key, and tested insert/update/delete operations for Kudu tables with non unique primary key. - Added front end and end to end unit tests. Passed query_test/test_kudu.py and custom_cluster/test_kudu.py on local environment with new version of Kudu built on local machine. - TODO build toolchian with new version of Kudu, including the commits for KUDU-1945. Run core test. Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd --- M common/thrift/CatalogObjects.thrift M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/TableDef.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/FeDb.java M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java M fe/src/main/jflex/sql-scanner.flex M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 40 files changed, 1,143 insertions(+), 197 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/10 -- To view, visit http://gerrit.cloudera.org:8080/19383 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd Gerrit-Change-Number: 19383 Gerrit-PatchSet: 10 Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Reviewer: Abhishek Chennaka <achenn...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <ale...@apache.org> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Marton Greber <greber...@gmail.com> Gerrit-Reviewer: Qifan Chen <qfc...@hotmail.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
