[
https://issues.apache.org/jira/browse/IMPALA-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106870#comment-17106870
]
ASF subversion and git services commented on IMPALA-6984:
---------------------------------------------------------
Commit 506a303c6d48fd5d0653cdfd58b3ae419328ff31 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=506a303 ]
IMPALA-6984: coordinator cancels backends on EOS
Before this patch, when the coordinator returned the last row,
it waited for backends to finish of their own accord, which
could happen indirectly as exchanges got closed.
The idea of this change is to send out cancellation RPCs to
expedite cancellation, then wait for the final exec status
reports to come in. Those reports will be included in the
final profile because the backend is *not* marked as done
when sending out the cancellation RPCs.
The bulk of this change is modifying the cancellation code
path to allow sending the cancel RPCs but *not* consider
the backend done until it gets back the final status report.
The old "fire and forget" mode of cancellation is still used
for explicit cancellation and errors.
Testing:
Ran exhaustive tests.
Ran cancellation tests under TSAN, checked for errors.
Manually inspected logs of some queries with limit,
saw that it sent cancellation then waited for backends
as expected.
Added a functional perf test that goes from ~5s down to < ~1s
on my system.
Change-Id: I966eceaafdc18a019708b780aee4ee9d70fd3a47
Reviewed-on: http://gerrit.cloudera.org:8080/15840
Reviewed-by: Thomas Tauber-Marshall <tmarsh...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Coordinator should cancel backends when returning EOS
> -----------------------------------------------------
>
> Key: IMPALA-6984
> URL: https://issues.apache.org/jira/browse/IMPALA-6984
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Affects Versions: Impala 3.0
> Reporter: Daniel Hecht
> Assignee: Tim Armstrong
> Priority: Major
> Labels: query-lifecycle
> Fix For: Impala 4.0
>
>
> Currently, the Coordinator waits for backends rather than proactively
> cancelling them in the case of hitting EOS. There's a tangled mess that makes
> it tricky to proactively cancel the backends related to how
> {{Coordinator::ComputeQuerySummary()}} works – we can't update the summary
> until the profiles are no longer changing (which also makes sense given that
> we want the exec summary to be consistent with the final profile). But we
> current tie together the FIS status and the profile, and cancellation of
> backends causes the FIS to return CANCELLED, which then means that the
> remaining FIS on that backend won't produce a final profile.
> With the rework of the protocol for IMPALA-2990 we should make it possible to
> sort this out such that a final profile can be requested regardless of how a
> FIS ends execution.
> This also relates to IMPALA-5783.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
