Paul Rogers has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12427
Change subject: IMPALA-8039: Incorrect selectivity estimate for not-equals predicate ...................................................................... IMPALA-8039: Incorrect selectivity estimate for not-equals predicate Impala has historically used a generic selectivity of 0.1 for all non-equality predicates. However, this can underestimate cardinality in some cases. The correct value is sel(c != x) = 1 - sel(c = x) if c is a column, x is a constant and ndv(c) is known. Adds the above for the != case and for IS DISTINCT FROM case. There are many related issues that are left as separate patches. Tests: * Used the newly-added expression cardinality tests to highlight the change (some formerly broken tests now pass). * Used the newly-added cardinality tests in PlannerTest to highlight the fix. * Fixed a merge artifact by having the PlannerTest cardinality tests actually verify cardinality. Change-Id: I8f6013c9ef95a89d55d8b25f0b5433c81582a62f --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-scan.test M testdata/workloads/functional-planner/queries/PlannerTest/hbase.test M testdata/workloads/functional-planner/queries/PlannerTest/inline-view-limit.test M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-views.test 12 files changed, 136 insertions(+), 114 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12427/2 -- To view, visit http://gerrit.cloudera.org:8080/12427 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8f6013c9ef95a89d55d8b25f0b5433c81582a62f Gerrit-Change-Number: 12427 Gerrit-PatchSet: 2 Gerrit-Owner: Paul Rogers <prog...@cloudera.com>
