Wenzhe Zhou has uploaded a new patch set (#21). ( http://gerrit.cloudera.org:8080/17553 )
Change subject: IMPALA-10557: Support Kudu's multi-row transaction ...................................................................... IMPALA-10557: Support Kudu's multi-row transaction Kudu added multi-row transaction so Impala could run query that inserts multiple rows into Kudu's table in the context of a single transaction. Kudu provides new Java/C++ client APIs to open/commit/rollback transaction, create session with transaction, serialize/deserialize metadata of transaction object. Kudu transaction object has built-in heartbeater. This patch added Impala support to use Kudu's multiple-row transaction. - Added a new query option to enable Kudu's transaction. - When the query option is set, a new Kudu transaction should be started for "insert", "CTAS" and "UPDATE/UPSERT/DELETE" statements by Impala's frontend of coordinator. - The Kudu transaction objects are kept in KuduTransactionManager until the transactions are going to be aborted or committed. - Frontend serialize the transaction metadata into a transaction token and pass to executors. - Executors deserialize the transaction token and ingest via that transaction handle. For Kudu session in the context of a transaction, return the first error if there are any pending errors for the Kudu session so that the Kudu transaction will be aborted. Since Kudu does not support transaction for "UPDATE/UPSERT/DELETE" statements now, Kudu returns error which causes transaction to be aborted. - Coordinator commits the transaction if everything goes well. Otherwise, aborts the transaction. Also changed code to store KuduClient as shared pointer since KuduClient has to be passed as a shared pointer when KuduTransaction::Deserialize() is called. Testing: - Added new e-to-e tests for Kudu transaction. - Passed core test. Change-Id: I876ada48991afdff5d61b5d6a0417571aba7cb34 --- M be/src/exec/kudu-scan-node-base.h M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h M be/src/exprs/kudu-partition-expr.cc M be/src/exprs/kudu-partition-expr.h M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/service/client-request-state.cc M be/src/service/client-request-state.h M be/src/service/frontend.cc M be/src/service/frontend.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/DataSinks.thrift M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/DeleteStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/analysis/UpdateStmt.java A fe/src/main/java/org/apache/impala/common/KuduTransactionManager.java M fe/src/main/java/org/apache/impala/common/TransactionException.java M fe/src/main/java/org/apache/impala/planner/KuduTableSink.java M fe/src/main/java/org/apache/impala/planner/TableSink.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl M tests/custom_cluster/test_kudu.py 29 files changed, 794 insertions(+), 67 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/53/17553/21 -- To view, visit http://gerrit.cloudera.org:8080/17553 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I876ada48991afdff5d61b5d6a0417571aba7cb34 Gerrit-Change-Number: 17553 Gerrit-PatchSet: 21 Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
