[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated PIG-3051: ------------------------------ Attachment: pig-3051-v1.1-withe2etest.txt Added an e2e test since I needed columnprune + limitoptimizer. Let me know if unit test is better for this. Also, appreciate if someone can take a look at my fix since this issue can lead to incorrect output. > java.lang.IndexOutOfBoundsException failure with LimitOptimizer + > ColumnPruning > -------------------------------------------------------------------------------- > > Key: PIG-3051 > URL: https://issues.apache.org/jira/browse/PIG-3051 > Project: Pig > Issue Type: Bug > Components: parser > Affects Versions: 0.10.0, 0.11 > Reporter: Koji Noguchi > Assignee: Koji Noguchi > Attachments: pig-3051-v1.1-withe2etest.txt, > pig-3051-v1-withouttest.txt > > > Had a user hitting > "Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1" error > when he had multiple stores and limit in his code. > I couldn't reproduce this with short pig code (due to ColumnPruning somehow > not happening when shortened), but here's a snippet. > {noformat} > ... > G3 = FOREACH G2 GENERATE sortCol, FLATTEN(group) as label, (long)COUNT(G1) as > cnt; > G4 = ORDER G3 BY cnt DESC PARALLEL 25; > ONEROW = LIMIT G4 1; > U1 = FOREACH ONEROW GENERATE 3 as sortcol, 'somelabel' as label, cnt; > store U1 into 'u1' using PigStorage(); > store G4 into 'g4' using PigStorage(); > {noformat} > With '-t ColumnMapKeyPrune', job didn't hit the error. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira