Re: Flink streaming. Broadcast reference data map across nodes

Hi, what Ufuk said is valid. In addition, you can make your function a RichFunction and load the static data in the open() method. In the future, you might be able to handle this use case with a feature called side inputs that we're currently working on:

Re: Flink streaming. Broadcast reference data map across nodes

On Tue, Feb 21, 2017 at 2:35 PM, Vadim Vararu wrote: > Basically, i have a big dictionary of reference data that has to be > accessible from all the nodes (in order to do some joins of log line with > reference line). If the dictionary is small you can make it part of

Flink streaming. Broadcast reference data map across nodes

Hi all, I would like to do something similar to Spark's broadcast mechanism. Basically, i have a big dictionary of reference data that has to be accessible from all the nodes (in order to do some joins of log line with reference line). I did not find yet a way to do it. Any ideas?