Stamatis Zampetakis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/25030
Change subject: IMPALA-15489: Inefficient CTE plans with trivial IS NOT NULL filters ...................................................................... IMPALA-15489: Inefficient CTE plans with trivial IS NOT NULL filters Move the NOT NULL derivation logic from joins after the CTE rewriting phase to avoid generating inefficient candidates and exploit more commonalities by avoiding the extra predicates making some sub-plans distinct. There are three category of plan changes. Join order change: Q13,Q58,Q64,Q68,Q77,Q93 Larger CTE detection: Q1,Q30,Q41,Q65,Q81 NOT NULL CTE removal: Q2,Q4,Q11,Q14a,Q23b,Q24b,Q31,Q38,Q39a,Q39b,Q44,Q46,Q56,Q58,Q60,Q64,Q66,Q68,Q70,Q74,Q76,Q77,Q80a,Q83,Q87,Q88,Q95 TODO: Add more details in the commit message Change-Id: Id5084b4b03cb0277a945338855a2d85f6c04a211 --- M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaCoreRules.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteOptimizer.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/planner/TpcdsCtePlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q01.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q03.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q04.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q05.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q06.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q07.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q10a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q11.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q15.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q17.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q18.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q21.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q22.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q23a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q23b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q25.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q27.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q29.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q31.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q32.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q34.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q35a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q36.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q38.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q39a.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q39b.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q40.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q43-verbose.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q43.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q45.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q46.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q47.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q49.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q50.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q51.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q53.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q54.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q57.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q58.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q59.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q62.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q63.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q64.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q65.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q67.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q68.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q69.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q70.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q72.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q73.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q74.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q76.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q77.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q78.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q79.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q83.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q86.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q87.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q88.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q89.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q90.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q92.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q93.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q95.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q96.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q97.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q98.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q99.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q01.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q02.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q03.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q04.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q05.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q06.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q10.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q11.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q14a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q14b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q15.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q17.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q18.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q21.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q22.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q23a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q23b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q25.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q27.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q29.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q31.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q32.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q34.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q35.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q36.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q38.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q39a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q39b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q40.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q41.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q45.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q46.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q47.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q49.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q50.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q51.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q57.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q58.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q59.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q62.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q64.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q65.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q67.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q68.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q69.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q70.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q72.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q73.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q74.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q76.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q77.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q78.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q79.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q80.test A testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q80a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q83.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q86.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q87.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q88.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q90.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q92.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q93.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q95.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q97.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q99.test 187 files changed, 71,909 insertions(+), 66,152 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/25030/1 -- To view, visit http://gerrit.cloudera.org:8080/25030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id5084b4b03cb0277a945338855a2d85f6c04a211 Gerrit-Change-Number: 25030 Gerrit-PatchSet: 1 Gerrit-Owner: Stamatis Zampetakis <[email protected]>
