[
https://issues.apache.org/jira/browse/DRILL-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8400:
--------------------------------
Fix Version/s: 1.21.0
> Fix pruning partitions with pushed transitive predicates
> --------------------------------------------------------
>
> Key: DRILL-8400
> URL: https://issues.apache.org/jira/browse/DRILL-8400
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.20.3
> Reporter: Vova Vysotskyi
> Assignee: Vova Vysotskyi
> Priority: Major
> Fix For: 1.21.0
>
>
> See {{TestHivePartitionPruning.prunePartitionsBasedOnTransitivePredicates()}}
> test for details.
> The issue occurs for queries like these:
> {code:sql}
> SELECT * FROM hive.partition_pruning_test t1
> JOIN hive.partition_with_few_schemas t2 ON t1.`d` = t2.`d` AND t1.`e` =
> t2.`e`
> WHERE t2.`e` IS NOT NULL AND t1.`d` = 1
> {code}
> The expected behavior is to create additional filters based on the existing
> filters and join conditions. We have a {{TRANSITIVE_CLOSURE}} planning phase,
> which is responsible for such query transformations, but Drill pushes down
> filters from the WHERE condition before that phase, so the optimization is
> not performed.
> Ideally, we should move rules from the {{TRANSITIVE_CLOSURE}} phase to the
> {{LOGICAL}} phase so that the planner will choose the most optimal plan, but
> it wouldn't help until CALCITE-1048 is fixed (it is required to pull
> predicates when three has {{RelSubset}} nodes).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
