Hello Tim Armstrong, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13051
to look at the new patch set (#3).
Change subject: IMPALA-7957: Fix slot equivalences may be enforced multiple
times
......................................................................
IMPALA-7957: Fix slot equivalences may be enforced multiple times
Predicates can be divided into three types according to the way they are
generated:
1) origin predicates that come from the query
2) auxiliary equal predicates generated for equivalence between a
label(alias) and its real expression
3) inferred predicates that inferred from the slot equivalences graph
The slot equivalences graph (valueTransferGraph in Analyzer) is
generated by the first two kinds of predicates. Analyzer will create
equivalence predicates for a PlanNode based on the unassigned predicates
and the valueTransferGraph. However, the current implementation can't
avoid creating inferred predicates that are duplicated with previously
created inferred predicates if they have been assigned before.
Duplicated inferred predicates are either redundant or wrong. Say, if we
create predicate p1: s1 = s2 for the current PlanNode and p1 duplicates
with a previously inferred predicate p0: s1 = s2 (same as s2 = s1), we
can prove that p1 is redundant or wrong:
1) p0 must have been assigned. Otherwise, p0 will be in the unassigned
conjuncts list and p1 won't be created.
2) p0 must have been assigned to an offspring node of the current
PlanNode since we create the PlanNodes in a depth first manner.
3) The origin predicates that infer to p0 have been assigned to an
offspring node too.
Then, rows that should be rejected have been filtered out either by p0
or the origin predicates that infer to p0. What's worse, assigning p1 on
top of the origin predicates may wrongly reject rows. Hence, p1 is
either redundant or wrong.
In this patch, we check the existence of previously inferred equivalence
predicates before creating a predicate. Also add some useful TRACE level
logs.
Tests:
* Add tests for UNIONs in inline-view.test
* Run all tests locally in CORE exploration strategy
Change-Id: Ida2d5d8149b217e18ebae61e136848162503653e
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M testdata/workloads/functional-planner/queries/PlannerTest/inline-view.test
5 files changed, 365 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/13051/3
--
To view, visit http://gerrit.cloudera.org:8080/13051
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ida2d5d8149b217e18ebae61e136848162503653e
Gerrit-Change-Number: 13051
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
