Hello Qifan Chen, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16120
to look at the new patch set (#7).
Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................
IMPALA-9903: Reduce Kudu openTable calls per query
This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.
It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.
Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.
Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.
In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.
Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 86 insertions(+), 32 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/7
--
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <granthe...@apache.org>
Gerrit-Reviewer: Grant Henke <granthe...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>
