Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12248 )
Change subject: IMPALA-8095: Detailed expression cardinality tests ...................................................................... IMPALA-8095: Detailed expression cardinality tests Cardinality is a critical input to the query planning process, especially join planning. Impala has many high-level end-to-end tests that implicitly test cardinality at the "wholesale" level: A test will produce a wrong result if the cardinality is badly wrong. This patch adds detailed unit tests for cardinality: * Table cardinality, NDV values and null count in metadata retrieved from HMS. * Table cardinality, NDV values and null counts in metadata presented to the query. * Expression NDV and selectivity values (which derive from table cardinality and column NDV.) The test illustrate a number of bugs. This patch simply identifies the bugs, comments out the tests that fail because of the bugs, and substitutes tests that pass with the current, incorrect, behavior. Future patches will fix the bugs. Reviewers can note the difference between the original, incorrect behavior shown here, and the revised behavior in those additional patches. Since none of the existing "functional" tables provide the level of detail needed for these tests, added a new test table specifically for this task. This set of tests was a good time to extend the test "fixture" framework created earlier. The FrontendTestBase class was refactored to use a new FrontendFixture which represents a (simulated) Impala and HMS cluster. The previous SessionFixture represents a single user session (with session options) and the QueryFixture represents a single query. As part of this refactoring, the fixture classes moved into "common" alongside FrontendTestBase. Testing: This patch includes only tests: no "production" code was changed. Change-Id: I3da58ee9b0beebeffb170b9430bd36d20dcd2401 Reviewed-on: http://gerrit.cloudera.org:8080/12248 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/analysis/SelectListItem.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java D fe/src/test/java/org/apache/impala/analysis/AnalysisSessionFixture.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java A fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/analysis/ExprNdvTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java A fe/src/test/java/org/apache/impala/common/AbstractFrontendTest.java A fe/src/test/java/org/apache/impala/common/AnalysisSessionFixture.java A fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java A testdata/NullRows/data.csv M testdata/bin/compute-table-stats.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv 17 files changed, 1,704 insertions(+), 533 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12248 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3da58ee9b0beebeffb170b9430bd36d20dcd2401 Gerrit-Change-Number: 12248 Gerrit-PatchSet: 9 Gerrit-Owner: Paul Rogers <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Fredy Wijaya <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Paul Rogers <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
