Nandor Kollar created PIG-5212:
----------------------------------
Summary: SkewedJoin_6 is failing on Spark
Key: PIG-5212
URL: https://issues.apache.org/jira/browse/PIG-5212
Project: Pig
Issue Type: Sub-task
Components: spark
Reporter: Nandor Kollar
Fix For: spark-branch
result are different:
{code}
diff <(head -20 SkewedJoin_6_benchmark.out/out_sorted) <(head -20
SkewedJoin_6.out/out_sorted)
< alice allen 19 1.930 alice allen 27 1.950
< alice allen 19 1.930 alice allen 34 1.230
< alice allen 19 1.930 alice allen 36 2.270
< alice allen 19 1.930 alice allen 38 0.810
< alice allen 19 1.930 alice allen 38 1.800
< alice allen 19 1.930 alice allen 42 2.460
< alice allen 19 1.930 alice allen 43 0.880
< alice allen 19 1.930 alice allen 45 2.800
< alice allen 19 1.930 alice allen 46 3.970
< alice allen 19 1.930 alice allen 51 1.080
< alice allen 19 1.930 alice allen 68 3.390
< alice allen 19 1.930 alice allen 68 3.510
< alice allen 19 1.930 alice allen 72 1.750
< alice allen 19 1.930 alice allen 72 3.630
< alice allen 19 1.930 alice allen 74 0.020
< alice allen 19 1.930 alice allen 74 2.400
< alice allen 19 1.930 alice allen 77 2.520
< alice allen 20 2.470 alice allen 27 1.950
< alice allen 20 2.470 alice allen 34 1.230
< alice allen 20 2.470 alice allen 36 2.270
---
> alice allen 27 1.950 alice allen 19 1.930
> alice allen 27 1.950 alice allen 20 2.470
> alice allen 27 1.950 alice allen 27 1.950
> alice allen 27 1.950 alice allen 34 1.230
> alice allen 27 1.950 alice allen 36 2.270
> alice allen 27 1.950 alice allen 38 0.810
> alice allen 27 1.950 alice allen 38 1.800
> alice allen 27 1.950 alice allen 42 2.460
> alice allen 27 1.950 alice allen 43 0.880
> alice allen 27 1.950 alice allen 45 2.800
> alice allen 27 1.950 alice allen 46 3.970
> alice allen 27 1.950 alice allen 51 1.080
> alice allen 27 1.950 alice allen 68 3.390
> alice allen 27 1.950 alice allen 68 3.510
> alice allen 27 1.950 alice allen 72 1.750
> alice allen 27 1.950 alice allen 72 3.630
> alice allen 27 1.950 alice allen 74 0.020
> alice allen 27 1.950 alice allen 74 2.400
> alice allen 27 1.950 alice allen 77 2.520
> alice allen 34 1.230 alice allen 19 1.930
{code}
It looks like the two tables are in wrong order, columns from 'a' should come
first, then columns from 'b'. In spark mode this is inverted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
