Hi all, I need to use the broadcast state mechanism (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) for the next scenario.
I have a reference data stream (slow) and an events stream (fast running) and I want to do a kind of lookup in the reference stream for each event. The broadcast state mechanism seems to fit perfect the scenario. >From documentation: As an example where broadcast state can emerge as a natural fit, one can imagine a low-throughput stream containing a set of rules which we want to evaluate against all elements coming from another stream. However, I am not sure what is the correct way to delay the consumption of the fast running stream until the slow one is fully read (in case of a file) or until a marker is emitted (in case of some other source). Is there any way to accomplish that? It doesn't seem to be a rare use case. Thanks, Vadim.
