Wenzhe Zhou has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20258 )
Change subject: IMPALA-12305: Fix wrong Catalog Service ID when CatalogD becomes active ...................................................................... IMPALA-12305: Fix wrong Catalog Service ID when CatalogD becomes active In IMPALA-12286, catalogd re-generate its Catalog Service ID in JniCatalog when it becomes active. But CatalogServiceCatalog is not updated when new Catalog Service ID is generated. This causes coordinator hanging when processing DDLs. In CatalogServer class, is_active_ is not protected by mutex catalog_lock_, and pending_topic_updates_ is not cleared when the catalogd becomes active. It's possible catalog server sends pending catalog topic updates with old Catalog Service ID then sends catalog topic updates with new Catalog Service ID. This patch removes catalogServiceId_ from CatalogServiceCatalog, and makes CatalogServiceCatalog to call JniCatalog.getServiceId() directly. It makes is_active_ to be protected by mutex catalog_lock_, and makes catalog server to clear pending_topic_updates_ when the catalogd becomes active. Testing: - Added more queries in test cases for CatalogD HA. Verified that test cases failed without fix, and test cases were passed after fix. - Passed core tests. Change-Id: I5eada89052c5f16209c6f16357f30f78b4497434 Reviewed-on: http://gerrit.cloudera.org:8080/20258 Reviewed-by: Andrew Sherman <[email protected]> Reviewed-by: Riza Suminto <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/catalog/catalog-server.cc M be/src/catalog/catalog-server.h M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/custom_cluster/test_catalogd_ha.py 7 files changed, 100 insertions(+), 84 deletions(-) Approvals: Andrew Sherman: Looks good to me, but someone else must approve Riza Suminto: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/20258 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5eada89052c5f16209c6f16357f30f78b4497434 Gerrit-Change-Number: 20258 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
