Hello Anurag Mantripragada, Vihang Karajgaonkar, Tim Armstrong, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15887
to look at the new patch set (#6).
Change subject: IMPALA-9669: Fix wrong types/comments of loaded tables/views
for GET_TABLES in LocalCatalog
......................................................................
IMPALA-9669: Fix wrong types/comments of loaded tables/views for GET_TABLES in
LocalCatalog
Coordinator can be in two modes: legacy mode or LocalCatalog mode.
Before IMPALA-8606, GET_TABLES required all tables to be loaded in
LocalCatalog-mode coordinator’s cache, which is a performance regression
compared to legacy mode coordinators. IMPALA-8606 changes the behavior
to only load the table names and create LocalIncompleteTable for each
table, which boosts the performance but results in all views being
returned with the default table type (TABLE). Besides this, all returned
comments are empty even if the table/view is loaded. This is a
regression since in legacy coordinators, loaded tables/views are shown
with correct table types and comments.
This is fixed by loading table types and comments along with table names
from catalogd. The cached list of table names of a DB is changed to be a
map containing the brief table metadata (name, type, comment). In case
of stale types or comments in the list, when loading the msTable of a
table, coordinator checks the type and comment and invalidates the table
list if any of them are stale.
Tests
- Add tests in test_hs2.test_get_tables
- Run CORE tests
Change-Id: I2180c603f061838347936f718cd4a0257d82e633
---
M common/thrift/CatalogService.thrift
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIncompleteTable.java
M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java
M fe/src/main/java/org/apache/impala/service/MetadataOp.java
M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java
M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java
M tests/hs2/test_hs2.py
11 files changed, 199 insertions(+), 68 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/15887/6
--
To view, visit http://gerrit.cloudera.org:8080/15887
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2180c603f061838347936f718cd4a0257d82e633
Gerrit-Change-Number: 15887
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Anurag Mantripragada <anu...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>
