[
https://issues.apache.org/jira/browse/PIG-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901641#comment-14901641
]
Daniel Dai commented on PIG-4683:
---------------------------------
[~rohini]
1) No, because currently SecondaryKeyOptimizerTez does not optimize cogroup.
But if we enable that in Tez, we will face the same issue.
2) PigSecondaryKeyComparator should only do sorting. Whether tuple containing
null should collapse together is the responsibility of
PigSecondaryKeyGroupComparator, which I didn't touch. Actually in the patch, I
only changed the code path which key doesn't contain null.
> Nested order is broken after PIG-3591 in some cases
> ---------------------------------------------------
>
> Key: PIG-4683
> URL: https://issues.apache.org/jira/browse/PIG-4683
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.16.0, 0.15.1
>
> Attachments: PIG-4683-1.patch
>
>
> The following script fail after PIG-3591.
> {code}
> a = load '1.txt' using PigStorage(',') as (a0:chararray, a1:chararray);
> b = load '2.txt' as (b0:chararray);
> c = cogroup b by b0, a by a0;
> d = foreach c {
> a_sorted = order a by a1 desc;
> generate group, a_sorted, b;
> }
> dump d;
> {code}
> 1.txt
> a,0
> a,2
> a,1
> 2.txt
> a
> Expected:
> {code}
> (a,{(a,1),(a,2),(a,0)},{(a)})
> {code}
> Actual:
> {code}
> (a,{(a,2),(a,1),(a,0),(a)},{})
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
