[
https://issues.apache.org/jira/browse/PIG-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901228#comment-14901228
]
Daniel Dai edited comment on PIG-4683 at 9/21/15 7:23 PM:
----------------------------------------------------------
The reason is PigSecondaryKeyComparator doesn't sort on index, but POPackage
assumes iterator is sorted by index after PIG-3591. Before that, if there are
multiple inputs, POPackage will create multiple bag and accumulate
individually. Now once POPackage sees an index of the last input, it stop
accumulating and create a PeekedBag immediately. However, the iterator may have
records of other inputs not processed.
was (Author: daijy):
The reason is PigSecondaryKeyComparator doesn't sort on index. But POPackage
assumes iterator is sorted by index after PIG-3591.
> Nested order is broken after PIG-3591 in some cases
> ---------------------------------------------------
>
> Key: PIG-4683
> URL: https://issues.apache.org/jira/browse/PIG-4683
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.16.0
>
>
> The following script fail after PIG-3591.
> {code}
> a = load '1.txt' using PigStorage(',') as (a0:chararray, a1:chararray);
> b = load '2.txt' as (b0:chararray);
> c = cogroup b by b0, a by a0;
> d = foreach c {
> a_sorted = order a by a1 desc;
> generate group, a_sorted, b;
> }
> dump d;
> {code}
> 1.txt
> a,0
> a,2
> a,1
> 2.txt
> a
> Expected:
> {code}
> (a,{(a,1),(a,2),(a,0)},{(a)})
> {code}
> Actual:
> {code}
> (a,{(a,2),(a,1),(a,0),(a)},{})
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
