I have a fairly complex (400 lines) pig script that fails on pig 8
(top of the branch), pig 9, and trunk and it consistently fails with
an error along the lines of the following:
2011-06-03 03:27:22,864 [main] ERROR org.apache.pig.tools.grunt.Grunt
- ERROR 2229: Couldn't find matching uid -1 for project (Name: Project
Type: bytearray Uid: 2341 Input: 0 Column: 1)
(full stack trace from Pig 8 under my signature).
Turning off ColumnMapKeyPrune makes the problem go away.
Unfortunately there is no line number or name of offending relation,
and the message is pretty obscure, so this is going to take a while to
debug.. are there known issues with this optimization or should I go
dig in?
D
Pig Stack Trace
---------------
ERROR 2229: Couldn't find matching uid -1 for project (Name: Project
Type: bytearray Uid: 2341 Input: 0 Column: 1)
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2042: Error
in new logical plan. Try -Dpig.usenewlogicalplan=false.
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:313)
at org.apache.pig.PigServer.compilePp(PigServer.java:1365)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1207)
at org.apache.pig.PigServer.execute(PigServer.java:1201)
at org.apache.pig.PigServer.access$100(PigServer.java:129)
at org.apache.pig.PigServer$Graph.execute(PigServer.java:1528)
at org.apache.pig.PigServer.executeBatchEx(PigServer.java:373)
at org.apache.pig.PigServer.executeBatch(PigServer.java:340)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:115)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:172)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:500)
at org.apache.pig.Main.main(Main.java:107)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
2000: Error processing rule ColumnMapKeyPrune. Try -t
ColumnMapKeyPrune
at
org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:281)
... 13 more
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
2229: Couldn't find matching uid -1 for project (Name: Project Type:
bytearray Uid: 2341 Input: 0 Column: 1)
at
org.apache.pig.newplan.logical.optimizer.ProjectionPatcher$ProjectionRewriter.visit(ProjectionPatcher.java:89)
at
org.apache.pig.newplan.logical.expression.ProjectExpression.accept(ProjectExpression.java:75)
at
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
at
org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at
org.apache.pig.newplan.logical.optimizer.AllExpressionVisitor.visit(AllExpressionVisitor.java:127)
at
org.apache.pig.newplan.logical.relational.LOInnerLoad.accept(LOInnerLoad.java:109)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at
org.apache.pig.newplan.logical.optimizer.AllExpressionVisitor.visit(AllExpressionVisitor.java:105)
at
org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:71)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at
org.apache.pig.newplan.logical.optimizer.ProjectionPatcher.transformed(ProjectionPatcher.java:48)
at
org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:113)
... 14 more
