Collect will help then . May be something like this, foreachRDD( rdd => { for(item <- rdd.collect().toArray) { println(item); } })
On Wed, Jul 6, 2016 at 5:46 PM, Sean Owen <so...@cloudera.com> wrote: > That's still causing the element to be printed on the remote > executors, not the driver. You'd have to collect the RDD and then > println, really. Also see DStream.print() > > On Wed, Jul 6, 2016 at 1:07 PM, Rabin Banerjee > <dev.rabin.baner...@gmail.com> wrote: > > It's not working because , you haven't collected the data. > > > > Try something like > > > > DStream.forEachRDD((rdd)=> {rdd.foreach(println)}) > > > > Thanks, > > Rabin > > > > > > On Wed, Jul 6, 2016 at 5:05 PM, Yu Wei <yu20...@hotmail.com> wrote: > >> > >> Hi guys, > >> > >> > >> It seemed that when launching application via yarn on single node, > >> JavaDStream.print() did not work. However, occasionally it worked. > >> > >> If launch the same application in local mode, it always worked. > >> > >> > >> The code is as below, > >> > >> SparkConf conf = new SparkConf().setAppName("Monitor&Control"); > >> JavaStreamingContext jssc = new JavaStreamingContext(conf, > >> Durations.seconds(1)); > >> JavaReceiverInputDStream<String> inputDS = MQTTUtils.createStream(jssc, > >> "tcp://114.55.145.185:1883", "Control"); > >> inputDS.print(); > >> jssc.start(); > >> jssc.awaitTermination(); > >> > >> > >> Command for launching via yarn, (did not work) > >> spark-submit --master yarn --deploy-mode cluster --driver-memory 4g > >> --executor-memory 2g target/CollAna-1.0-SNAPSHOT.jar > >> > >> Command for launching via local mode (works) > >> spark-submit --master local[4] --driver-memory 4g --executor-memory 2g > >> --num-executors 4 target/CollAna-1.0-SNAPSHOT.jar > >> > >> > >> Any thoughts about the problem? > >> > >> > >> Thanks, > >> > >> Jared > >> > > >