Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20276
Change subject: IMPALA-12321: Fix the race condition for updating active catalogd ...................................................................... IMPALA-12321: Fix the race condition for updating active catalogd When CatalogD HA is enabled, statestored sends the address of current active catalogd to coordinators and catalogds during registration, or a new active catalogd is elected. Statestored sends the active catalogd in two different kinds of RPCs. If there are more than one election changes in short time, coordinators and catalogds could receive RPCs in the order which are different from the changing order on statestore. To make coordinators and catalogds to have same view as statestore, we have to avoid to overwrite the latest version of active catalogd with previous change. Version of active catalogd is added in UpdateCatalogd RPC, but not in response message of statestore registration. This patch adds active catalogd version in the response message of statestore registration. Coordinators and catalogds only apply the changes which have newer version than the version of last recevied active catalogd. The version of last received active catalogd have to be re-synced for new registration since statestore could be restarted. Allow subscribers to skip UpdateCatalogd RPC if subscribers cannot handle it, for example the statestore-id is unknown for a subscriber since the RPC is received before registration is completed. The skipped RPCs will be resent by statestore. Testing: - Added a test case to start both catalogds with flag 'force_catalogd_active' as true. - Passed core tests Change-Id: Ie49947e563d43c59bdd476b28c35be69848ae12a --- M be/src/catalog/catalog-server.cc M be/src/catalog/catalog-server.h M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/statestore/statestore-catalogd-mgr.cc M be/src/statestore/statestore-catalogd-mgr.h M be/src/statestore/statestore-subscriber.cc M be/src/statestore/statestore-subscriber.h M be/src/statestore/statestore.cc M be/src/statestore/statestore.h M common/thrift/StatestoreService.thrift M tests/custom_cluster/test_catalogd_ha.py 12 files changed, 272 insertions(+), 135 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/20276/1 -- To view, visit http://gerrit.cloudera.org:8080/20276 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie49947e563d43c59bdd476b28c35be69848ae12a Gerrit-Change-Number: 20276 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
