Broadcast copies arbitrary objects, so you could read it into an object such an array of lines then broadcast that.
Andy On Monday, 17 November 2014, YaoPau <jonrgr...@gmail.com> wrote: > I have a 1 million row file that I'd like to read from my edge node, and > then > send a copy of it to each Hadoop machine's memory in order to run JOINs in > my spark streaming code. > > I see examples in the docs of how use use broadcast() for a simple array, > but how about when the data is in a textFile? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-broadcast-a-textFile-tp19083.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org <javascript:;> > For additional commands, e-mail: user-h...@spark.apache.org <javascript:;> > >
