Fabian Hueske created FLINK-2318:
------------------------------------
Summary: BroadcastVariable of unioned data set fails
Key: FLINK-2318
URL: https://issues.apache.org/jira/browse/FLINK-2318
Project: Flink
Issue Type: Bug
Components: Distributed Runtime, Optimizer
Affects Versions: 0.9
Reporter: Fabian Hueske
Using a unioned data set as broadcast variable such as this:
{code}
DataSet d1 = [...]
DataSet d2 = [...]
DataSet d3 = [...]
d1
.map(new MyMapper())
.withBroadcastSet(d2.union(d3), "myBroadcast");
{code}
throws an exception at runtime:
{code}
java.lang.Exception: Call to registerInputOutput() of invokable failed
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:504)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Initializing the input streams failed in
Task MapPartition (MapPartition at
translatHashJoinAsMap(FlinkFlowStep.java:755)): Illegal input group size in
task configuration: -1
at
org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:246)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:501)
... 1 more
Caused by: java.lang.Exception: Illegal input group size in task configuration:
-1
at
org.apache.flink.runtime.operators.RegularPactTask.initBroadcastInputReaders(RegularPactTask.java:783)
at
org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:243)
... 2 more
{code}
A simple workaround is to apply an identity mapper on the unioned data set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
