Hi Marco,
(a) It is possible for an operator to receive two different kind of broadcasts,
DataStream ints =
DataStream strs = ...
ints.broadcast().connect(strs.broadcast())
.process(new CoProcessFunction(){...});
(b) Traditional Flink operator could not accept three different inputs.
There is a new MultipleInputOperator that could accept arbitrary number
of inputs [1]. However It is currently not expose directly to end users,
and you would need to work on some low-level api to use it. Or an alternative
might be use a tag to union the two input streams (or any two of the three
inputs)
and use the (keyed)CoProcessFunction above. Also note that the broadcast is
only a partitioner,
and it is treated no difference with other partitioners for downstream
operators.
Best,
Yun
[1]
https://github.com/apache/flink/blob/51524de8fd337aafd30952873b36216c5a3c43bc/flink-streaming-java/src/test/java/org/apache/flink/streaming/api/graph/StreamGraphGeneratorTest.java#L261
--
Sender:Marco Villalobos
Date:2021/03/06 09:47:53
Recipient:user
Theme:questions about broadcasts
Is it possible for an operator to receive two different kinds of broadcasts?
Is it possible for an operator to receive two different types of streams and a
broadcast? For example, I know there is a KeyedCoProcessFunction, but is there
a version of that which can also receive broadcasts?
