[
https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925201#action_12925201
]
Namit Jain commented on HIVE-1750:
----------------------------------
I think we can use the following rules:
If the predicates contain only ANDs, remove the predicate containing
partitioning columns
In case of any UDF (including OR):
Get the columns used as arguments
If all columns in the parameters are partitioning columns, we can remove the
partitioning predicate.
If a column in the partitioning predicate contains a non-partitioning column,
we cannot remove it,
for eg:
If the condition is:
ds=1 or ds=2
ds=1 and x=1
we can remove the conditions for partitioning columns.
However, if the condition is:
ds = 1 or x = 1
we cannot modify the condition.
We can go over all UDFs and add them in the category of UDFs which behave like
AND.
Any unknowns behave like OR
> Remove Partition Filtering Conditions when Possible
> ---------------------------------------------------
>
> Key: HIVE-1750
> URL: https://issues.apache.org/jira/browse/HIVE-1750
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Siying Dong
>
> For some simple queries, partition filtering constraints take 8% of CPU time
> (now 16% since we filter twice) even if the result is always true. When
> possible, we should remove these constraints to save CPU times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
