[ https://issues.apache.org/jira/browse/PIG-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886907#action_12886907 ]
Scott Carey commented on PIG-1491: ---------------------------------- Full stack trace at the end of this comment. The pig script used is a couple hundred lines long. But the individual chunk that I can change to trigger the issue is the following: DATA_G = COGROUP A by (a, b, c, d) OUTER, B by (a, b, c, d) OUTER; DATA = FOREACH DATA_G { a_items = DISTINCT A.x; b_items = DISTINCT B.x; GENERATE FLATTEN(group) as (a,b,c,d), SUM(A.m) as m, SUM(A.n) as n, COUNT(a_items) as a_item_count, (long)(SUM(B.u) + (double)0.5) as u, (long)(SUM(B.v) + (double)0.5) as v, COUNT(b_items) as b_item_count; } Removing both of the DISTINCT temporary aliases and not generating those counts works fine. Adding either one of them causes it to fail. ERROR 2043: Unexpected error during execution. org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution. at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:318) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1016) at org.apache.pig.PigServer.execute(PigServer.java:1009) at org.apache.pig.PigServer.access$100(PigServer.java:114) at org.apache.pig.PigServer$Graph.execute(PigServer.java:1261) at org.apache.pig.PigServer.executeBatch(PigServer.java:326) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:110) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:167) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:139) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90) at org.apache.pig.Main.main(Main.java:335) Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:218) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:40) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50) at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:446) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:108) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:294) ... 10 more > Failure planning nested FOREACH with DISTINCT, POLoad cannot be cast to > POLocalRearrange > ---------------------------------------------------------------------------------------- > > Key: PIG-1491 > URL: https://issues.apache.org/jira/browse/PIG-1491 > Project: Pig > Issue Type: Bug > Affects Versions: 0.7.0 > Reporter: Scott Carey > > I have a failure that occurs during planning while using DISTINCT in a nested > FOREACH. > Caused by: java.lang.ClassCastException: > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad > cannot be cast to > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:218) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:40) > at > org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.