[
https://issues.apache.org/jira/browse/PIG-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15961945#comment-15961945
]
Daniel Dai commented on PIG-5211:
---------------------------------
Another comment, in OptimizeNestedLimitTransformer, when we iterator through
innerPlan.getOperators(), we cannot assume the order of operators in the
iterator, so the check (pred instanceof LOSort && op instanceof LOLimit) is not
always valid. We can find the limit operator first, and then use
currentPlan.getPredecessors(limit) to make sure the predecessor is LOSort.
> Optimize Nested Limited Sort
> ----------------------------
>
> Key: PIG-5211
> URL: https://issues.apache.org/jira/browse/PIG-5211
> Project: Pig
> Issue Type: Improvement
> Reporter: Jin Sun
> Assignee: Jin Sun
> Fix For: 0.17.0
>
> Attachments: PIG-5211-1.patch, PIG-5211-2.patch, PIG-5211-3.patch
>
>
> Currently in FOREACH clause, if both LIMIT and ORDER BY are present, pig
> stores all elements and sort them. It should use a priority queue to be more
> efficient in space.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
