[
https://issues.apache.org/jira/browse/PIG-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315883#comment-14315883
]
Bing Jiang commented on PIG-3594:
---------------------------------
[~aniket486], checking the source code of pig-3461, it does not cover the case.
Tracking the reason, RegexExpression cannot be accepted by
PartitionFilterOptimization. And your test case just verifies that the
RegexExpression does not impact the pushdown of other partitions. I understand
it is reasonable to disallow the RegexExpression to join
PartitionFilterOptimization, but could we make sure that other valid partitions
can be pushdown to LOAD operation?
> Pig 0.11 with HCatLoader partition filter does not push down valid conditions
> -----------------------------------------------------------------------------
>
> Key: PIG-3594
> URL: https://issues.apache.org/jira/browse/PIG-3594
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11.1
> Reporter: Mona Chitnis
> Fix For: 0.11.2
>
>
> When running the following pig query (script.pig) :
> A = LOAD ' my_data' USING org.apache.hcatalog.pig.HCatLoader();
> B = FILTER A by part1=='$val1' and part2 matches '$val2';
> C = FILTER B by map#'key' matches '(val3|val4)';
> DUMP C;
> > pig -useHCatalog -p val1=2013_01_01 val2=XYZ script.pig
> This results in Pig scanning all partitions inspite of filter condition.
> part1=='$val1' should have been pushed to HCatLoader even though there are
> other conditions that cannot be pushed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
