Hi I'm trying to use broadcast variables in my Spark streaming program. val conf = new SparkConf().setMaster(SPARK_MASTER).setAppName(APPLICATION_NAME) val ssc = new StreamingContext(conf, Seconds(1)) val LIMIT = ssc.sparkContext.broadcast(5L) println(LIMIT.value) // this print 5 val lines = ssc.socketTextStream("localhost", 9999) val words = lines.flatMap(_.split(" ")) filter (_.size < LIMIT.value) words.print() ssc.start() ssc.awaitTermination()
It throws java.lang.NullPointerException at the line (_.size < LIMIT.value) so I'm guessing LIMIT is not accessible within the stream I'm running spark 1.3.1 in standalone mode with 2 nodes cluster. I tried with spark-shell and it works fine. Please help! Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/NullPointerException-when-accessing-broadcast-variable-in-DStream-tp22934.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org