Hi Jon, You'll need to put the file on HDFS (or whatever distributed filesystem you're running on) and load it from there.
-Sandy On Thu, Feb 5, 2015 at 3:18 PM, YaoPau <jonrgr...@gmail.com> wrote: > I have a file "badFullIPs.csv" of bad IP addresses used for filtering. In > yarn-client mode, I simply read it off the edge node, transform it, and > then > broadcast it: > > val badIPs = fromFile(edgeDir + "badfullIPs.csv") > val badIPsLines = badIPs.getLines > val badIpSet = badIPsLines.toSet > val badIPsBC = sc.broadcast(badIpSet) > badIPs.close > > How can I accomplish this in yarn-cluster mode? > > Jon > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-broadcast-a-variable-read-from-a-file-in-yarn-cluster-mode-tp21524.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
