[
https://issues.apache.org/jira/browse/PIG-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644711#comment-13644711
]
Rohini Palaniswamy commented on PIG-3173:
-----------------------------------------
> But this patch at least lets Pig push down expressions when A, B, C, and D
> are ALL partition conditions. IMO, this alone is a big win. Can we get this
> patch in and do further optimization on the '(A and B) or (C and D)' case in
> a separate jira?
Sure. I had started on the optimization patch but I did not complete it
before leaving for vacation. Wanted to be careful as there was lot of change
and I had to almost evaluate the whole tree and ensure it works for all
combinations as we are extracting partial conditions. I will create a separate
jira for that and put the patch later. I will update this patch (first one had
a bug) with just pushing down all partition conditions.
> Partition filter push down does not happen partition keys condition include a
> AND and OR construct
> --------------------------------------------------------------------------------------------------
>
> Key: PIG-3173
> URL: https://issues.apache.org/jira/browse/PIG-3173
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.1
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.12
>
> Attachments: PIG-3173-1.patch
>
>
> A = load 'db.table' using org.apache.hcatalog.pig.HCatLoader();
> B = filter A by (region=='usa' AND dt=='201302051800') OR (region=='uk' AND
> dt=='201302051800');
> C = foreach B generate name, age;
> DUMP C;
> gives the below warning and scans the whole table.
> 2013-02-06 22:22:16,233 [main] WARN
> org.apache.pig.newplan.PColFilterExtractor - No partition filter push down:
> You have an partition column (region ) in a construction like: (pcond and
> ...) or (pcond and ...) where pcond is a condition on a partition column.
> 2013-02-06 22:22:16,233 [main] WARN
> org.apache.pig.newplan.PColFilterExtractor - No partition filter push down:
> You have an partition column (datestamp ) in a construction like: (pcond and
> ...) or (pcond and ...) where pcond is a condition on a partition column.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
