Hi, I am trying simple file streaming example using Sparkstreaming(spark-streaming_2.10,version:1.5.1)
public class DStreamExample { public static void main(final String[] args) { final SparkConf sparkConf = new SparkConf(); sparkConf.setAppName("SparkJob"); sparkConf.setMaster("local[4]"); // for local final JavaSparkContext sc = new JavaSparkContext(sparkConf); final JavaStreamingContext ssc = new JavaStreamingContext(sc, new Duration(2000)); final JavaDStream<String> lines = ssc.textFileStream("/opt/test/"); lines.print(); ssc.start(); ssc.awaitTermination(); } } When I run this code on single file or director it does not print anything from file, I see in logs its constantly polling but nothing is printed. I tried moving file to directory when this program was running. Is there something I am missing? I tried applying map function on lines RDD that also does not work. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spak-filestreaming-issue-tp25380.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org