PIG Logical Optimization: Push LOFilter above LOCogroup
--------------------------------------------------------

                 Key: PIG-1530
                 URL: https://issues.apache.org/jira/browse/PIG-1530
             Project: Pig
          Issue Type: New Feature
          Components: impl
            Reporter: Swati Jain
            Assignee: Swati Jain
            Priority: Minor
             Fix For: 0.8.0


Consider the following:
{noformat}
A = load '<any file>' USING PigStorage(',') as (a1:int,a2:int,a3:int);
B = load '<any file>' USING PigStorage(',') as (b1:int,b2:int,b3:int);
G = COGROUP A by (a1,a2) , B by (b1,b2);
D = Filter G by group.$0 + 5 > group.$1;
explain D;
{noformat}

In the above example, LOFilter can be pushed above LOCogroup. Note there are 
some tricky NULL issues to think about when the Cogroup is not of type INNER 
(Similar to issues that need to be thought through when pushing LOFilter on the 
right side of a LeftOuterJoin).

Also note that typically the LOFilter in user programs will be below a 
ForEach-Cogroup pair. To make this really useful, we need to also implement 
LOFilter pushed across ForEach. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to