[
https://issues.apache.org/jira/browse/PIG-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708551#action_12708551
]
Pradeep Kamath commented on PIG-802:
------------------------------------
Adding some more details:
A new kind of bag - ReadOnceBag needs to be implemented. This bag will have
reference to the "key" currently being processed and the iterator to values
provided by hadoop in reduce(). The ReadOnceBag's iterator will simply iterate
over the hadoop iterator at each call and construct a tuple by using the key
and value (see POPackage.java for details on how this is done). POPackage
should also be changed or a new class introduced which creates ReadOnceBags
instead of regular bags. This creation of the bag should only initialize the
bag with the key and iterator.
> PERFORMANCE: not creating bags for ORDER BY
> -------------------------------------------
>
> Key: PIG-802
> URL: https://issues.apache.org/jira/browse/PIG-802
> Project: Pig
> Issue Type: Improvement
> Reporter: Olga Natkovich
>
> Order by should be changed to not use POPackage to put all of the tuples in a
> bag on the reduce side, as the bag is just immediately flattened. It can
> instead work like join does for the last input in the join.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
