[
https://issues.apache.org/jira/browse/PIG-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-761:
---------------------------
Attachment: PIG-761-1.patch
The problem lies in the complexity between limit and one of the optimization.
More specific, optimization POPackageAnnotator search for matching
POLocalRearrange in the map plan, if not, search in the predecessor's reduce
plan. However, if we have a limit, limit will introduce a map-reduce job
between the original map-reduce job and its predecessor. POPackageAnnotator
cannot find the POLocalRearrange then. To fix it, we mark the map reduce job
introduced by limit, and in POPackageAnnotator, if we saw a limit map reduce
job, we will search POLocalRearrange in limit job's parent.
> ERROR 2086 on simple JOIN
> -------------------------
>
> Key: PIG-761
> URL: https://issues.apache.org/jira/browse/PIG-761
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.2.0
> Environment: mapreduce mode
> Reporter: Vadim Zaliva
> Fix For: 0.6.0
>
> Attachments: PIG-761-1.patch
>
>
> ERROR 2086: Unexpected problem during optimization. Could not find all
> LocalRearrange operators.org.apache.pig.impl.logicalLayer.FrontendException:
> ERROR 1002: Unable to store alias 109
> doing pretty straightforward join in one of my pig scripts. I am able to
> 'dump' both relationship involved in this join. when I try to join them I am
> getting this error.
> Here is a full log:
> ERROR 2086: Unexpected problem during optimization. Could not find all
> LocalRearrange operators.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable
> to store alias 109
> at org.apache.pig.PigServer.registerQuery(PigServer.java:296)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:529)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:280)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
> at org.apache.pig.Main.main(Main.java:319)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
> 2043: Unexpected error during execution.
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:274)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:700)
> at org.apache.pig.PigServer.execute(PigServer.java:691)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:292)
> ... 5 more
> Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException:
> ERROR 2086: Unexpected problem during optimization. Could not find all
> LocalRearrange operators.
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans.POPackageAnnotator.handlePackage(POPackageAnnotator.java:116)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans.POPackageAnnotator.visitMROp(POPackageAnnotator.java:88)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:194)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:43)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:65)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
> MapReduceLauncher.compile(MapReduceLauncher.java:198)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:80)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:261)
> ... 8 more
> ERROR 1002: Unable to store alias 398
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable
> to store alias 398
> at org.apache.pig.PigServer.registerQuery(PigServer.java:296)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:529)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:280)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
> at org.apache.pig.Main.main(Main.java:319)
> Caused by: java.lang.NullPointerException
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:669)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:330)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:41)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:246)
> at org.apache.pig.PigServer.compilePp(PigServer.java:771)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:697)
> at org.apache.pig.PigServer.execute(PigServer.java:691)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:292)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
