[
https://issues.apache.org/jira/browse/PIG-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-4479:
------------------------------------
Attachment: PIG-4479-1.patch
This is a special case where multiple LocalRearrange in POSplit point to same
vertex group. We usually avoid keeping them in POSplit, but the LocalRearranges
are in different nested splits. Not changing MultiQueryOptimizer to avoid this
as well as vertex group handles multiple edges from same vertex unlike regular
edge.
> Pig script with many nested splits failed on Tez
> ------------------------------------------------
>
> Key: PIG-4479
> URL: https://issues.apache.org/jira/browse/PIG-4479
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.15.0
>
> Attachments: PIG-4479-1.patch
>
>
> {code}
> ERROR 2087: Unexpected problem during optimization. Found index:0 in multiple
> LocalRearrange operators.
> org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2087: Unexpected
> problem during optimization. Found index:0 in multiple LocalRearrange
> operators.
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.TezPOPackageAnnotator$LoRearrangeDiscoverer.visitLocalRearrange(TezPOPackageAnnotator.java:170)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.visit(POLocalRearrange.java:185)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.visit(POLocalRearrange.java:52)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhyPlanVisitor.visitSplit(PhyPlanVisitor.java:168)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
