[
https://issues.apache.org/jira/browse/PIG-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105879#comment-14105879
]
Lorand Bendig commented on PIG-4135:
------------------------------------
Yes, that's the intended use case. I suddenly was thinking of some other
examples but you are correct, using filter should be the cleanest and safest
way to pick those map-only jobs that can be quickly fetched from grunt. So +1,
thank you for the fix!
> Fetch optimization should be disabled if plan contains no limit
> ---------------------------------------------------------------
>
> Key: PIG-4135
> URL: https://issues.apache.org/jira/browse/PIG-4135
> Project: Pig
> Issue Type: Bug
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.14.0
>
> Attachments: PIG-4135-1.patch
>
>
> After deploying fetch optimization in production, a couple of users ran into
> this situation. They had fairly large input data, but after filtering it by a
> regular expression, it becomes small. So they didn't add limit to the query.
> The problem is that even though the output is small, processing the input
> must be done in the cluster not in the client. However, fetch optimization
> blindly fetches the entire input into the client since the plan is map-only
> job and finishes with dump.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
