Have you considered using external storage such as hbase for storing the look up table ?
Cheers On Fri, Oct 2, 2015 at 11:50 AM, <saif.a.ell...@wellsfargo.com> wrote: > I tried broadcasting a key-value rdd, but then I cannot perform any > rdd-actions inside a map/foreach function of another rdd. > > any tips? If going into scala collections I end up with huge memory > bottlenecks. > > Saif > >
