[
https://issues.apache.org/jira/browse/FLINK-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643764#comment-15643764
]
Chesnay Schepler commented on FLINK-2662:
-----------------------------------------
[~nielsbasjes] reported another instance of this issue, which can be reproduced
with the following code:
{quote}
The smallest code snippet I have been able to create that reproduces this
problem is below here.
Note that when using a single union this error does not happen.
{quote}
{code}
public class Main implements Serializable {
public static void main(String[] args) throws Exception {
System.exit(new Main().run());
}
private int run() throws IOException {
final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
final DataSet<String> lines =
env.createInput(new TextInputFormat(new
Path("/tmp/doesNotExist")))
.union(env.createInput(new TextInputFormat(new
Path("/tmp/doesNotExist"))))
.union(env.createInput(new TextInputFormat(new
Path("/tmp/doesNotExist"))));
List<String> allLines = new ArrayList<>();
lines
.rebalance()
.output(new LocalCollectionOutputFormat<>(allLines));
// execute program
try {
env.execute("Running");
} catch (Exception e) {
e.printStackTrace();
}
return 0;
}
}
{code}
> CompilerException: "Bug: Plan generation for Unions picked a ship strategy
> between binary plan operators."
> ----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-2662
> URL: https://issues.apache.org/jira/browse/FLINK-2662
> Project: Flink
> Issue Type: Bug
> Components: Optimizer
> Affects Versions: 0.9.1, 0.10.0
> Reporter: Gabor Gevay
> Assignee: Fabian Hueske
> Priority: Critical
> Fix For: 1.0.0, 1.2.0, 1.1.3
>
> Attachments: Bug.java, FlinkBug.scala
>
>
> I have a Flink program which throws the exception in the jira title. Full
> text:
> Exception in thread "main" org.apache.flink.optimizer.CompilerException: Bug:
> Plan generation for Unions picked a ship strategy between binary plan
> operators.
> at
> org.apache.flink.optimizer.traversals.BinaryUnionReplacer.collect(BinaryUnionReplacer.java:113)
> at
> org.apache.flink.optimizer.traversals.BinaryUnionReplacer.postVisit(BinaryUnionReplacer.java:72)
> at
> org.apache.flink.optimizer.traversals.BinaryUnionReplacer.postVisit(BinaryUnionReplacer.java:41)
> at
> org.apache.flink.optimizer.plan.DualInputPlanNode.accept(DualInputPlanNode.java:170)
> at
> org.apache.flink.optimizer.plan.SingleInputPlanNode.accept(SingleInputPlanNode.java:199)
> at
> org.apache.flink.optimizer.plan.DualInputPlanNode.accept(DualInputPlanNode.java:163)
> at
> org.apache.flink.optimizer.plan.DualInputPlanNode.accept(DualInputPlanNode.java:163)
> at
> org.apache.flink.optimizer.plan.WorksetIterationPlanNode.acceptForStepFunction(WorksetIterationPlanNode.java:194)
> at
> org.apache.flink.optimizer.traversals.BinaryUnionReplacer.preVisit(BinaryUnionReplacer.java:49)
> at
> org.apache.flink.optimizer.traversals.BinaryUnionReplacer.preVisit(BinaryUnionReplacer.java:41)
> at
> org.apache.flink.optimizer.plan.DualInputPlanNode.accept(DualInputPlanNode.java:162)
> at
> org.apache.flink.optimizer.plan.SingleInputPlanNode.accept(SingleInputPlanNode.java:199)
> at
> org.apache.flink.optimizer.plan.SingleInputPlanNode.accept(SingleInputPlanNode.java:199)
> at
> org.apache.flink.optimizer.plan.OptimizedPlan.accept(OptimizedPlan.java:127)
> at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:520)
> at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:402)
> at
> org.apache.flink.client.LocalExecutor.getOptimizerPlanAsJSON(LocalExecutor.java:202)
> at
> org.apache.flink.api.java.LocalEnvironment.getExecutionPlan(LocalEnvironment.java:63)
> at malom.Solver.main(Solver.java:66)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> The execution plan:
> http://compalg.inf.elte.hu/~ggevay/flink/plan_3_4_0_0_without_verif.txt
> (I obtained this by commenting out the line that throws the exception)
> The code is here:
> https://github.com/ggevay/flink/tree/plan-generation-bug
> The class to run is "Solver". It needs a command line argument, which is a
> directory where it would write output. (On first run, it generates some
> lookuptables for a few minutes, which are then placed to /tmp/movegen)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
