Paul Rogers has uploaded this change for review. (
http://gerrit.cloudera.org:8080/12427
Change subject: IMPALA-8039: Incorrect selectivity estimate for not-equals
predicate
..
IMPALA-8039: Incorrect selectivity estimate for not-equals predicate
Impala has historically used a generic selectivity of 0.1 for all
non-equality predicates. However, this can underestimate cardinality
in some cases. The correct value is
sel(c != x) = 1 - sel(c = x)
if c is a column, x is a constant and ndv(c) is known.
Adds the above for the != case and for IS DISTINCT FROM case. There are
many related issues that are left as separate patches.
Tests:
* Used the newly-added expression cardinality tests to highlight the
change (some formerly broken tests now pass).
* Used the newly-added cardinality tests in PlannerTest to highlight the
fix.
* Fixed a merge artifact by having the PlannerTest cardinality tests
actually verify cardinality.
Change-Id: I8f6013c9ef95a89d55d8b25f0b5433c81582a62f
---
M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java
M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/workloads/functional-planner/queries/PlannerTest/card-scan.test
M testdata/workloads/functional-planner/queries/PlannerTest/hbase.test
M
testdata/workloads/functional-planner/queries/PlannerTest/inline-view-limit.test
M
testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-views.test
12 files changed, 136 insertions(+), 114 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12427/2
--
To view, visit http://gerrit.cloudera.org:8080/12427
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8f6013c9ef95a89d55d8b25f0b5433c81582a62f
Gerrit-Change-Number: 12427
Gerrit-PatchSet: 2
Gerrit-Owner: Paul Rogers
