[
https://issues.apache.org/jira/browse/PIG-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated PIG-1574:
-----------------------------
Status: Patch Available (was: Open)
> Optimization rule PushUpFilter causes filter to be pushed up out joins
> ----------------------------------------------------------------------
>
> Key: PIG-1574
> URL: https://issues.apache.org/jira/browse/PIG-1574
> Project: Pig
> Issue Type: Bug
> Reporter: Xuefu Zhang
> Assignee: Xuefu Zhang
> Fix For: 0.8.0
>
> Attachments: jira-1574-1.patch
>
>
> The PushUpFilter optimization rule in the new logical plan moves the filter
> up to one of the join branch. It does this aggressively by find an operator
> that has all the projection UIDs. However, it didn't consider that the found
> operator might be another join. If that join is outer, then we cannot simply
> move the filter to one of its branches.
> As an example, the following script will be erroneously optimized:
> A = load 'myfile' as (d1:int);
> B = load 'anotherfile' as (d2:int);
> C = join A by d1 full outer, B by d2;
> D = load 'xxx' as (d3:int);
> E = join C by d1, D by d3;
> F = filter E by d1 > 5;
> G = store F into 'dummy';
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
