godfrey he created FLINK-12371: ---------------------------------- Summary: Add support for converting (NOT) IN/ (NOT) EXISTS to SemiJoin, and generating optimized logical plan Key: FLINK-12371 URL: https://issues.apache.org/jira/browse/FLINK-12371 Project: Flink Issue Type: New Feature Components: Table SQL / Planner Reporter: godfrey he Assignee: godfrey he
This issue aims to convert IN/EXISTS to semi-join, and NOT IN/NOT EXISTS to anti-join. In Calcite, [SemiJoin|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/SemiJoin.java] only represents semi-join, (could not represent anti-join) and requires equi join condition. Queries like `select * from left where left.a1 in (select right.a2 from right where left.b1 > right.b2)` and `select * from left where not exists (select * from right)` could not be converted to Calcite SemiJoin operator. To solve the above problem, We need copy the {{SemiJoin}} class to Flink, and make the following changes: 1. make {{SemiJoin}} extending from {{Join}}, not from {{EquiJoin}}. (to support non-equi join condition) 2. add {{isAnti}} field attribute to represent anti-join. Currently, there are no rules to convert (NOT) IN/ (NOT) EXISTS to SemiJoin, so we need a whole new set of rules to meet our requirement. 1. {{FlinkSubQueryRemoveRule}}, a planner rule that converts IN and EXISTS into semi-join, converts NOT IN and NOT EXISTS into anti-join 2. {{FlinkRewriteSubQueryRule}}, a planner rule that converts filter condition like: {{(select count(*) from T) > 0}} to {{exists(select * from T)}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)