[
https://issues.apache.org/jira/browse/PIG-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543420#comment-15543420
]
Daniel Dai commented on PIG-5033:
---------------------------------
scope-61 definitely cause the issue as we cannot have parallel edges between a
pair of vertex. The patch prevent multiquery optimization once detecting
replicated join. However, I think the proper fix should detect parallel edges.
[~rohini], would you like to take a look?
> MultiQueryOptimizerTez creates bad plan with union, split and FRJoin
> --------------------------------------------------------------------
>
> Key: PIG-5033
> URL: https://issues.apache.org/jira/browse/PIG-5033
> Project: Pig
> Issue Type: Bug
> Components: tez
> Affects Versions: 0.16.0
> Reporter: Travis Woodruff
> Attachments: PIG-5033.patch, input1, input2, input3
>
>
> This script produces incorrect results:
> {code}
> a = load 'file:///tmp/input1' as (x:int, y:int);
> b = load 'file:///tmp/input2' as (x:int, y:int);
> u = union a,b;
> c = load 'file:///tmp/input3' as (x:int, y:int);
> e = filter c by y > 3;
> f = filter c by y < 2;
> g = join u by x left, e by x using 'replicated';
> h = join g by u::x left, f by x using 'replicated';
> store h into 'file:///tmp/pigoutput';
> {code}
> Without the union, or with opt.multiquery=false, or with non-replicated
> joins, it works as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
