You can't broadcast an RDD to begin with, and can't use RDDs inside RDDs. They are really driver-side concepts.
Yes that's how you'd use a broadcast of anything else though, though you need to reference ".value" on the broadcast. The 'if' is redundant in that example, and if it's a map- or collection-like structure, you don't even need the arg. RDD2.filter(broadcasted.value.contains) On Mon, Dec 7, 2015 at 2:43 PM, Akhil Das <ak...@sigmoidanalytics.com> wrote: > Something like this? > > val broadcasted = sc.broadcast(...) > > RDD2.filter(value => { > > //simply use broadcasted > if(broadcasted.contains(value)) true > > }) > > > > Thanks > Best Regards > > On Fri, Dec 4, 2015 at 10:43 PM, Abhishek Shivkumar > <abhishek.shivku...@bigdatapartnership.com> wrote: >> >> Hi, >> >> I have RDD1 that is broadcasted. >> >> I have a user defined method for the filter functionality of RDD2, written >> as follows: >> >> RDD2.filter(my_func) >> >> >> I want to access the values of RDD1 inside my_func. Is that possible? >> Should I pass RDD1 as a parameter into my_func? >> >> Thanks >> Abhishek S >> >> NOTICE AND DISCLAIMER >> >> This email (including attachments) is confidential. If you are not the >> intended recipient, notify the sender immediately, delete this email from >> your system and do not disclose or use for any purpose. >> >> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United >> Kingdom >> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United >> Kingdom >> Big Data Partnership Limited is a company registered in England & Wales >> with Company No 7904824 > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org