Re: Streaming - lookup against reference data

2016-09-15 Thread Tom Davis
e. I guess for a > true evaluation you have to check your use case, the envisioned future > architecture for other use cases, required performance, maintability etc. > > On 14 Sep 2016, at 20:44, Tom Davis <mailinglists...@gmail.com> wrote: > > Hi all, > > Interested i

Streaming - lookup against reference data

2016-09-14 Thread Tom Davis
Hi all, Interested in patterns people use in the wild for lookup against reference data sets from a Spark streaming job. The reference dataset will be updated during the life of the job (although being 30mins out of date wouldn't be an issue, for example). So far I have come up with a few