[
https://issues.apache.org/jira/browse/PIG-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644683#comment-13644683
]
Cheolsoo Park commented on PIG-3173:
------------------------------------
If I understand comments on RB, there is no real issue with the patch other
than that we can do better on the '(A and B) or (C and D)' case.
Currently, Pig rejects all of the following expressions even if A, B, C, and D
are all partition conditions:
- (A and B) or (C and D)
- (A and B) or C
- A or (C and D)
But this patch at least lets Pig push down expressions when A, B, C, and D are
ALL partition conditions. IMO, this alone is a big win. Can we get this patch
in and do further optimization on the '(A and B) or (C and D)' case in a
separate jira?
Thanks!
> Partition filter push down does not happen partition keys condition include a
> AND and OR construct
> --------------------------------------------------------------------------------------------------
>
> Key: PIG-3173
> URL: https://issues.apache.org/jira/browse/PIG-3173
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.1
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.12
>
> Attachments: PIG-3173-1.patch
>
>
> A = load 'db.table' using org.apache.hcatalog.pig.HCatLoader();
> B = filter A by (region=='usa' AND dt=='201302051800') OR (region=='uk' AND
> dt=='201302051800');
> C = foreach B generate name, age;
> DUMP C;
> gives the below warning and scans the whole table.
> 2013-02-06 22:22:16,233 [main] WARN
> org.apache.pig.newplan.PColFilterExtractor - No partition filter push down:
> You have an partition column (region ) in a construction like: (pcond and
> ...) or (pcond and ...) where pcond is a condition on a partition column.
> 2013-02-06 22:22:16,233 [main] WARN
> org.apache.pig.newplan.PColFilterExtractor - No partition filter push down:
> You have an partition column (datestamp ) in a construction like: (pcond and
> ...) or (pcond and ...) where pcond is a condition on a partition column.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
