[
https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498150#comment-13498150
]
Koji Noguchi commented on PIG-3051:
-----------------------------------
Stack dump from task failure using trunk
{noformat}
2012-11-09 22:45:09,484 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : org.apache.pig.backend.executionengine.ExecException:
ERROR 0: Error while executing ForEach at []
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:306)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit.getNext(POLimit.java:117)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:263)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:185)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:163)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:51)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170) at
org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1615)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1567)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1416)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:663)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:730)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at org.apache.pig.data.DefaultTuple.get(DefaultTuple.java:116)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage.getValueTuple(POPackage.java:345)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackageLite.getValueTuple(POPackageLite.java:198)
at
org.apache.pig.data.ReadOnceBag$ReadOnceBagIterator.next(ReadOnceBag.java:241)
at
org.apache.pig.data.ReadOnceBag$ReadOnceBagIterator.next(ReadOnceBag.java:222)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:440)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:298)
... 19 more
{noformat}
> java.lang.IndexOutOfBoundsException failure with LimitOptimizer +
> ColumnPruning
> --------------------------------------------------------------------------------
>
> Key: PIG-3051
> URL: https://issues.apache.org/jira/browse/PIG-3051
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.10.0, 0.11
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
>
> Had a user hitting
> "Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1" error
> when he had multiple stores and limit in his code.
> I couldn't reproduce this with short pig code (due to ColumnPruning somehow
> not happening when shortened), but here's a snippet.
> {noformat}
> ...
> G3 = FOREACH G2 GENERATE sortCol, FLATTEN(group) as label, (long)COUNT(G1) as
> cnt;
> G4 = ORDER G3 BY cnt DESC PARALLEL 25;
> ONEROW = LIMIT G4 1;
> U1 = FOREACH ONEROW GENERATE 3 as sortcol, 'somelabel' as label, cnt;
> store U1 into 'u1' using PigStorage();
> store G4 into 'g4' using PigStorage();
> {noformat}
> With '-t ColumnMapKeyPrune', job didn't hit the error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
