Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/20768 )
Change subject: IMPALA-12375: Make DataSource Object persistent ...................................................................... IMPALA-12375: Make DataSource Object persistent DataSource objects are saved in-memory cache in Catalog server. They are not persisted to the HMS. The objects are lost after Catalog server is restarted. This patch makes DataSource Object persistent by saving DataSource objects as DataConnector objects with type "impalaDataSource" in HMS. Since HMS events for DataConnector are not handled, Catalog server has to refresh DataSource objects when the catalogd becomes active. Testing: - Added end-to-end test cases with restarting of Catalog server, and catalogd HA failover. - Passed core test. Change-Id: I500a99142bb62ce873e693d573064ad4ffa153ab --- M be/src/catalog/catalog-server.cc M be/src/catalog/catalog.cc M be/src/catalog/catalog.h M fe/src/main/java/org/apache/impala/analysis/CreateTableDataSrcStmt.java M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/DataSource.java M fe/src/main/java/org/apache/impala/catalog/DataSourceTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M tests/custom_cluster/test_ext_data_sources.py 11 files changed, 271 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/20768/4 -- To view, visit http://gerrit.cloudera.org:8080/20768 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I500a99142bb62ce873e693d573064ad4ffa153ab Gerrit-Change-Number: 20768 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Anonymous Coward <gsi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
