[
https://issues.apache.org/jira/browse/PIG-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965241#comment-15965241
]
liyunzhang_intel commented on PIG-5212:
---------------------------------------
[~nkollar]: PIG-5212.patch is available and all unit tests pass. I describe the
changes of the patch in above comments,help review.
> SkewedJoin_6 is failing on Spark
> --------------------------------
>
> Key: PIG-5212
> URL: https://issues.apache.org/jira/browse/PIG-5212
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Nandor Kollar
> Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: PIG-5212.patch
>
>
> result are different:
> {code}
> diff <(head -20 SkewedJoin_6_benchmark.out/out_sorted) <(head -20
> SkewedJoin_6.out/out_sorted)
> < alice allen 19 1.930 alice allen 27 1.950
> < alice allen 19 1.930 alice allen 34 1.230
> < alice allen 19 1.930 alice allen 36 2.270
> < alice allen 19 1.930 alice allen 38 0.810
> < alice allen 19 1.930 alice allen 38 1.800
> < alice allen 19 1.930 alice allen 42 2.460
> < alice allen 19 1.930 alice allen 43 0.880
> < alice allen 19 1.930 alice allen 45 2.800
> < alice allen 19 1.930 alice allen 46 3.970
> < alice allen 19 1.930 alice allen 51 1.080
> < alice allen 19 1.930 alice allen 68 3.390
> < alice allen 19 1.930 alice allen 68 3.510
> < alice allen 19 1.930 alice allen 72 1.750
> < alice allen 19 1.930 alice allen 72 3.630
> < alice allen 19 1.930 alice allen 74 0.020
> < alice allen 19 1.930 alice allen 74 2.400
> < alice allen 19 1.930 alice allen 77 2.520
> < alice allen 20 2.470 alice allen 27 1.950
> < alice allen 20 2.470 alice allen 34 1.230
> < alice allen 20 2.470 alice allen 36 2.270
> ---
> > alice allen 27 1.950 alice allen 19 1.930
> > alice allen 27 1.950 alice allen 20 2.470
> > alice allen 27 1.950 alice allen 27 1.950
> > alice allen 27 1.950 alice allen 34 1.230
> > alice allen 27 1.950 alice allen 36 2.270
> > alice allen 27 1.950 alice allen 38 0.810
> > alice allen 27 1.950 alice allen 38 1.800
> > alice allen 27 1.950 alice allen 42 2.460
> > alice allen 27 1.950 alice allen 43 0.880
> > alice allen 27 1.950 alice allen 45 2.800
> > alice allen 27 1.950 alice allen 46 3.970
> > alice allen 27 1.950 alice allen 51 1.080
> > alice allen 27 1.950 alice allen 68 3.390
> > alice allen 27 1.950 alice allen 68 3.510
> > alice allen 27 1.950 alice allen 72 1.750
> > alice allen 27 1.950 alice allen 72 3.630
> > alice allen 27 1.950 alice allen 74 0.020
> > alice allen 27 1.950 alice allen 74 2.400
> > alice allen 27 1.950 alice allen 77 2.520
> > alice allen 34 1.230 alice allen 19 1.930
> {code}
> It looks like the two tables are in wrong order, columns from 'a' should come
> first, then columns from 'b'. In spark mode this is inverted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
