Stamatis Zampetakis created HIVE-28924:
------------------------------------------
Summary: Drop
hive.optimize.join.disjunctive.transitive.predicates.pushdown and related logic
Key: HIVE-28924
URL: https://issues.apache.org/jira/browse/HIVE-28924
Project: Hive
Issue Type: Task
Components: CBO
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
The hive.optimize.join.disjunctive.transitive.predicates.pushdown property was
introduced by HIVE-25758 to prevent the push-down of disjunctive predicates
since it was deemed risky and in some cases led to HS2 OOM.
The main cause behind the OOM was that the RexSimplify couldn't simplify some
predicates with disjunctions. However, after upgrading Calcite version to
1.33.0 (HIVE-27102), and the introduction of the SEARCH operator, the
simplifier is able to handle many more cases with disjunctions.
At the moment, all the test cases introduced by HIVE-25758 pass and transitive
predicate push-down kicks-in no matter the value of the property. In fact,
after HIVE-27102,
[DisjunctivePredicatesFinder|https://github.com/apache/hive/blob/28abf17424e97112d75947f1d7a2f043b3961fa4/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L1235]
returns false for most (if not all) test cases that utilize this code.
Given that all known problems that lead to OOM have been resolved, we can drop
the hive.optimize.join.disjunctive.transitive.predicates.pushdown property and
remove the code that handles the disjunctive predicates.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
