[
https://issues.apache.org/jira/browse/PIG-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886907#action_12886907
]
Scott Carey commented on PIG-1491:
----------------------------------
Full stack trace at the end of this comment.
The pig script used is a couple hundred lines long. But the individual chunk
that I can change to trigger the issue is the following:
DATA_G = COGROUP A by (a, b, c, d) OUTER, B by (a, b, c, d) OUTER;
DATA = FOREACH DATA_G {
a_items = DISTINCT A.x;
b_items = DISTINCT B.x;
GENERATE
FLATTEN(group) as (a,b,c,d),
SUM(A.m) as m,
SUM(A.n) as n,
COUNT(a_items) as a_item_count,
(long)(SUM(B.u) + (double)0.5) as u,
(long)(SUM(B.v) + (double)0.5) as v,
COUNT(b_items) as b_item_count;
}
Removing both of the DISTINCT temporary aliases and not generating those counts
works fine. Adding either one of them causes it to fail.
ERROR 2043: Unexpected error during execution.
org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected
error during execution.
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:318)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1016)
at org.apache.pig.PigServer.execute(PigServer.java:1009)
at org.apache.pig.PigServer.access$100(PigServer.java:114)
at org.apache.pig.PigServer$Graph.execute(PigServer.java:1261)
at org.apache.pig.PigServer.executeBatch(PigServer.java:326)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:110)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:167)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:139)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.main(Main.java:335)
Caused by: java.lang.ClassCastException:
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad
cannot be cast to
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:218)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:40)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:446)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:108)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:294)
... 10 more
> Failure planning nested FOREACH with DISTINCT, POLoad cannot be cast to
> POLocalRearrange
> ----------------------------------------------------------------------------------------
>
> Key: PIG-1491
> URL: https://issues.apache.org/jira/browse/PIG-1491
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Scott Carey
>
> I have a failure that occurs during planning while using DISTINCT in a nested
> FOREACH.
> Caused by: java.lang.ClassCastException:
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad
> cannot be cast to
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:218)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:40)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
