Hi Mohit,
do you plan to implement a batch or streaming job? If it is a streaming
job: You can use a connected stream (see [1], Slide 34). The static data
is one side of the stream that could be updated from time to time and
will always propagated (using a broadcast()) to all workers that do
filtering, augmentation etc.
[1] http://training.data-artisans.com/dataStream/1-intro.html
I hope this helps.
Timo
Am 13.07.17 um 02:16 schrieb Mohit Anchlia:
What is the best way to read a map of lookup data? This lookup data is
like a small short lived data that is available in transformation to
do things like filtering, additional augmentation of data etc.
