Hello Thomas Tauber-Marshall, Sahil Takiar, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15840
to look at the new patch set (#4).
Change subject: IMPALA-6984: coordinator cancels backends on EOS
......................................................................
IMPALA-6984: coordinator cancels backends on EOS
Before this patch, when the coordinator returned the last row,
it waited for backends to finish of their own accord, which
could happen indirectly as exchanges got closed.
The idea of this change is to send out cancellation RPCs to
expedite cancellation, then wait for the final exec status
reports to come in. Those reports will be included in the
final profile because the backend is *not* marked as done
when sending out the cancellation RPCs.
The bulk of this change is modifying the cancellation code
path to allow sending the cancel RPCs but *not* consider
the backend done until it gets back the final status report.
The old "fire and forget" mode of cancellation is still used
for explicit cancellation and errors.
Testing:
Ran exhaustive tests.
Ran cancellation tests under TSAN, checked for errors.
Manually inspected logs of some queries with limit,
saw that it sent cancellation then waited for backends
as expected.
Added a functional perf test that goes from ~5s down to < ~1s
on my system.
Change-Id: I966eceaafdc18a019708b780aee4ee9d70fd3a47
---
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator-backend-state.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/util/counting-barrier.h
A testdata/workloads/targeted-perf/queries/primitive_cancel_union.test
M tests/experiments/test_targeted_perf.py
7 files changed, 118 insertions(+), 36 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15840/4
--
To view, visit http://gerrit.cloudera.org:8080/15840
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I966eceaafdc18a019708b780aee4ee9d70fd3a47
Gerrit-Change-Number: 15840
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com>
