You can't broadcast an RDD to begin with, and can't use RDDs inside
RDDs. They are really driver-side concepts.
Yes that's how you'd use a broadcast of anything else though, though
you need to reference ".value" on the broadcast. The 'if' is redundant
in that example, and if it's a map- or
Something like this?
val broadcasted = sc.broadcast(...)
RDD2.filter(value => {
//simply use *broadcasted*
if(broadcasted.contains(value)) true
})
Thanks
Best Regards
On Fri, Dec 4, 2015 at 10:43 PM, Abhishek Shivkumar <
abhishek.shivku...@bigdatapartnership.com> wrote:
> Hi,
>
> I have
Hi,
I have RDD1 that is broadcasted.
I have a user defined method for the filter functionality of RDD2, written
as follows:
RDD2.filter(my_func)
I want to access the values of RDD1 inside my_func. Is that possible?
Should I pass RDD1 as a parameter into my_func?
Thanks
Abhishek S
--
