Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16266 )
Change subject: IMPALA-5022: Outer join simplification ...................................................................... Patch Set 12: (10 comments) Looks good! Thanks a lot to convince me on null-rejecting predicates being applied on non-join columns. On the tests. It seems most of them are Planner ones. I feel we probably should add some that can verify the correct-ness on transformation. One way to verify is to run a query with and without the transformation and compare the result. This also brings the need to add a query option such as enable_outer_join_to_inner_transformation (default to true). http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3286 PS12, Line 3286: containing nit: contains http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3286 PS12, Line 3286: the nit: Reword to 'if a' http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3287 PS12, Line 3287: is nit: reword to ", it is" http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3313 PS12, Line 3313: is nit: extra space. need to put 'the' before null-rejecting predicate. http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3482 PS12, Line 3482: case INNER_JOIN: { : if (tableRef.getLeftTblRef() != null) { : boolean ret = simplifyOuterJoinsByIjOnClause(processedTblRefs, tableRef); : isSimplified = isSimplified ? true : ret; : } : break; I wonder if we need this case here. http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java: http://gerrit.cloudera.org:8080/#/c/16266/12/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@327 PS12, Line 327: f We may need to add some more to the list, such as 'case'. See https://impala.apache.org/docs/build/html/topics/impala_conditional_functions.html. http://gerrit.cloudera.org:8080/#/c/16266/10/testdata/workloads/functional-planner/queries/PlannerTest/card-outer-join.test File testdata/workloads/functional-planner/queries/PlannerTest/card-outer-join.test: http://gerrit.cloudera.org:8080/#/c/16266/10/testdata/workloads/functional-planner/queries/PlannerTest/card-outer-join.test@95 PS10, Line 95: t outer join tpch.orders o on c.c_custkey = o.o_custkey : where c.c_name = 'foo' > c.c_name = 'foo' proves that the tuple c can't have null values, so we can OK. I agree the null rejecting predicate does not have to be on the join column. http://gerrit.cloudera.org:8080/#/c/16266/10/testdata/workloads/functional-planner/queries/PlannerTest/outer-to-inner-joins.test File testdata/workloads/functional-planner/queries/PlannerTest/outer-to-inner-joins.test: http://gerrit.cloudera.org:8080/#/c/16266/10/testdata/workloads/functional-planner/queries/PlannerTest/outer-to-inner-joins.test@52 PS10, Line 52: one of the t2's columns is not null the disjunctive conjunct is true, so this : # is null-rejecting conjunct > For t1.int_col + t2.int_col < 10, it maybe true only t2.int_col is not null OK. Here we have two null-rejecting predicates on t2. More than enough. http://gerrit.cloudera.org:8080/#/c/16266/10/testdata/workloads/functional-planner/queries/PlannerTest/outer-to-inner-joins.test@242 PS10, Line 242: d = t2.id > t2.zip can be null but t3.test_zip can't be null. This is null-rejecting pr OK. http://gerrit.cloudera.org:8080/#/c/16266/10/testdata/workloads/functional-planner/queries/PlannerTest/outer-to-inner-joins.test@293 PS10, Line 293: > same comment as above. Done -- To view, visit http://gerrit.cloudera.org:8080/16266 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa7804033fac68e93f33c387dc68ef67f803e93e Gerrit-Change-Number: 16266 Gerrit-PatchSet: 12 Gerrit-Owner: Xianqing He <hexianqing...@126.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Shant Hovsepian <sh...@cloudera.com> Gerrit-Reviewer: Xianqing He <hexianqing...@126.com> Gerrit-Comment-Date: Tue, 25 Aug 2020 14:35:57 +0000 Gerrit-HasComments: Yes
