[
https://issues.apache.org/jira/browse/PIG-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404023#comment-13404023
]
Jie Li commented on PIG-2661:
-----------------------------
{quote}
I see what you mean now – yeah, our options in this case are to either not
perform the optimization, or to push the operator chain above the sample
loader, and sample its outputs instead of its inputs.
{quote}
I think simply not performing the optimization is better, as the order-by
doesn't need to re-parse the data (see the results above); also it's easier to
implement.
{quote}
But on the flatten thing, it occurs to me that we actually shouldn't allow the
merge join here, as we can't guarantee sorted order after a flatten. Or can we?
Is there a reason to believe order will stay the same after a flatten?
{quote}
The order-by is after the flatten, thus the flatten shouldn't affect the final
order, right?
> Pig uses an extra job for loading data in Pigmix L9
> ---------------------------------------------------
>
> Key: PIG-2661
> URL: https://issues.apache.org/jira/browse/PIG-2661
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.9.0
> Reporter: Jie Li
> Assignee: Jie Li
> Attachments: PIG-2661.0.patch, PIG-2661.1.patch, PIG-2661.2.patch,
> PIG-2661.3.patch, PIG-2661.plan.txt
>
>
> See
> https://issues.apache.org/jira/browse/PIG-200?focusedCommentId=13260155&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13260155
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
