[ https://issues.apache.org/jira/browse/PIG-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-4135: ------------------------------- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thank you Daniel and Lorand for the review! > Fetch optimization should be disabled if plan contains no limit > --------------------------------------------------------------- > > Key: PIG-4135 > URL: https://issues.apache.org/jira/browse/PIG-4135 > Project: Pig > Issue Type: Bug > Reporter: Cheolsoo Park > Assignee: Cheolsoo Park > Fix For: 0.14.0 > > Attachments: PIG-4135-1.patch > > > After deploying fetch optimization in production, a couple of users ran into > this situation. They had fairly large input data, but after filtering it by a > regular expression, it becomes small. So they didn't add limit to the query. > The problem is that even though the output is small, processing the input > must be done in the cluster not in the client. However, fetch optimization > blindly fetches the entire input into the client since the plan is map-only > job and finishes with dump. -- This message was sent by Atlassian JIRA (v6.2#6252)