Hello Joe McDonnell, Csaba Ringhofer, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22994
to look at the new patch set (#3).
Change subject: IMPALA-14130: Wait catalog.curr-version in
start-impala-cluster.py
......................................................................
IMPALA-14130: Wait catalog.curr-version in start-impala-cluster.py
IMPALA-13850 changed the behavior of bin/start-impala-cluster.py to wait
for the number of tables to be at least one. This is needed to detect
that the catalog has seen at least one update. There is special logic in
dataload to start Impala without tables in that circumstance.
This broke the perf-AB-test job, which starts Impala before loading
data. There are other times when we want to start Impala without tables,
and it is inconvenient to need to specify --wait_num_tables each time.
It is actually not necessary to wait for catalog metric of Coordinator
to reach certain value. Frontend (Coordinator) will not open its service
port until it heard the first catalog topic update form CatalogD.
IMPALA-13850 (part 2) also ensure that CatalogD with
--catalog_topic_mode=minimal will block serving Coordinator request
until it begin its first reset() operation. Therefore, waiting
Coordinator's catalog version is not needed anymore and
--wait_num_tables parameter can be removed.
This patch also slightly change the "progress log" of
start-impala-cluster.py to print the Coordinator's catalog version
instead of num DB and tables cached. The sleep interval time now include
time spent checking Coordinator's metric.
Testing:
- Pass dataload with updated script.
- Manually run start-impala-cluster.py in both legacy and local catalog
mode and confirm it works.
- Pass custom cluster test_concurrent_ddls.py and test_catalogd_ha.py
Change-Id: I4a3956417ec83de4fb3fc2ef1e72eb3641099f02
---
M bin/start-impala-cluster.py
M testdata/bin/create-load-data.sh
M tests/common/impala_cluster.py
3 files changed, 14 insertions(+), 29 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/22994/3
--
To view, visit http://gerrit.cloudera.org:8080/22994
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a3956417ec83de4fb3fc2ef1e72eb3641099f02
Gerrit-Change-Number: 22994
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
