[
https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Noguchi updated PIG-3051:
------------------------------
Attachment: pig-3051-v2.1-withe2etest.txt
Thanks Rohini for the review.
bq .But found an issue with the copy not setting the label, type and Uid.
I wasn't sure why my test worked even when the above fields were not set. Turns
out that they are filled by the SchemaPatcher.
LimitOptimizer.reportChanges() simply returns currentPlan, so SchemaPatcher
goes through the entire currentPlan including the newSort.mSortColPlans
mentioned and update them accordingly.
BTW, reading back my patch, I felt that logic of making a new copy of LOSort
should be kept inside LOSort.java. Uploading a new version. Logic is same
from the previous patch.
> java.lang.IndexOutOfBoundsException failure with LimitOptimizer +
> ColumnPruning
> --------------------------------------------------------------------------------
>
> Key: PIG-3051
> URL: https://issues.apache.org/jira/browse/PIG-3051
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.10.0, 0.11
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Fix For: 0.11
>
> Attachments: pig-3051-v1.1-withe2etest.txt,
> pig-3051-v1-withouttest.txt, pig-3051-v2.1-withe2etest.txt
>
>
> Had a user hitting
> "Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1" error
> when he had multiple stores and limit in his code.
> I couldn't reproduce this with short pig code (due to ColumnPruning somehow
> not happening when shortened), but here's a snippet.
> {noformat}
> ...
> G3 = FOREACH G2 GENERATE sortCol, FLATTEN(group) as label, (long)COUNT(G1) as
> cnt;
> G4 = ORDER G3 BY cnt DESC PARALLEL 25;
> ONEROW = LIMIT G4 1;
> U1 = FOREACH ONEROW GENERATE 3 as sortcol, 'somelabel' as label, cnt;
> store U1 into 'u1' using PigStorage();
> store G4 into 'g4' using PigStorage();
> {noformat}
> With '-t ColumnMapKeyPrune', job didn't hit the error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
