Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10289 )
Change subject: IMPALA-6948: Delete catalog update topic entries upon catalog restart ...................................................................... IMPALA-6948: Delete catalog update topic entries upon catalog restart This commit fixes an issue where the statestore may end up with stale entries in the catalog update topic that do not correspond to the catalog objects stored in the catalog. This may occur if the catalog server restarts and some catalog object (e.g. table) that was known to the catalog before the restart no longer exists in the Hive Metastore after the restart. Fix: The first update for the catalog update topic that is sent by the catalog instructs the statestore to clear any entries it may have on this topic before applying the first update. In that way, we guarantee that the statestore entries are consistent with the catalog objects stored in the catalog server. Any coordinator that detects the catalog restart will receive from the statestore a full topic update that reflects the state of the catalog server. Testing: Added statestore test. Change-Id: I907509bf92da631ece5efd23c275a613ead00e91 Tmp Change-Id: I74a8ade8e498ac35cb56d3775d2c67a86988d9b6 Reviewed-on: http://gerrit.cloudera.org:8080/10289 Reviewed-by: Vuk Ercegovac <vercego...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/catalog/catalog-server.cc M be/src/statestore/statestore.cc M be/src/statestore/statestore.h M common/thrift/StatestoreService.thrift M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M tests/statestore/test_statestore.py 6 files changed, 98 insertions(+), 7 deletions(-) Approvals: Vuk Ercegovac: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10289 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I74a8ade8e498ac35cb56d3775d2c67a86988d9b6 Gerrit-Change-Number: 10289 Gerrit-PatchSet: 5 Gerrit-Owner: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com>
