[
https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siying Dong updated HIVE-1750:
------------------------------
Attachment: HIVE-1750.1.patch
After generating partition predicates, we add another step to prune partitions
and according to the list of pruned partitions, remove partition predicates
when possible. A new optimizer (PartitionConditionRemover) is created for it,
which walks the operator tree and the expression tree for target filter
operator. For every expression including partition column, we evaluate the
value for every partition pruned. If the result agrees for all partitions, we
replace expression to be the constant result. We further remove the nodes when
possible. Finally, if the whole expression tree is always to return true, we
remove the filter operator.
We only handle the filter pushed down to near the table scan operators. We'll
reply on HIVE-1538 to remove the original one.
Fix the unit test results accordingly and add a new unit test.
> Remove Partition Filtering Conditions when Possible
> ---------------------------------------------------
>
> Key: HIVE-1750
> URL: https://issues.apache.org/jira/browse/HIVE-1750
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Siying Dong
> Attachments: HIVE-1750.1.patch
>
>
> For some simple queries, partition filtering constraints take 8% of CPU time
> (now 16% since we filter twice) even if the result is always true. When
> possible, we should remove these constraints to save CPU times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
