Banerjee; user@spark.apache.org
Subject: Re: It seemed JavaDStream.print() did not work when launching via yarn
on a single node
DStream.print() will collect some of the data to driver and display, please see
the implementation of DStream.print()
RDD.take() will collect some of the data to
July 6, 2016 8:20:36 PM
> *To:* Rabin Banerjee
> *Cc:* Yu Wei; user@spark.apache.org
> *Subject:* Re: It seemed JavaDStream.print() did not work when launching
> via yarn on a single node
>
> dstream.foreachRDD(_.collect.foreach(println))
>
> On Wed, Jul 6, 2016 at 1:19 PM, Rabin Bane
How about DStream.print().
Does it invoke collect before print on driver?
From: Sean Owen
Sent: Wednesday, July 6, 2016 8:20:36 PM
To: Rabin Banerjee
Cc: Yu Wei; user@spark.apache.org
Subject: Re: It seemed JavaDStream.print() did not work when launching via
dstream.foreachRDD(_.collect.foreach(println))
On Wed, Jul 6, 2016 at 1:19 PM, Rabin Banerjee
wrote:
> Collect will help then . May be something like this,
> foreachRDD( rdd => { for(item <- rdd.collect().toArray) { println(item); }
> })
>
Collect will help then . May be something like this,
foreachRDD( rdd => { for(item <- rdd.collect().toArray) { println(item); }
})
On Wed, Jul 6, 2016 at 5:46 PM, Sean Owen wrote:
> That's still causing the element to be printed on the remote
> executors, not the driver. You'd have to collect th
That's still causing the element to be printed on the remote
executors, not the driver. You'd have to collect the RDD and then
println, really. Also see DStream.print()
On Wed, Jul 6, 2016 at 1:07 PM, Rabin Banerjee
wrote:
> It's not working because , you haven't collected the data.
>
> Try somet
It's not working because , you haven't collected the data.
Try something like
DStream.forEachRDD((rdd)=> {rdd.foreach(println)})
Thanks,
Rabin
On Wed, Jul 6, 2016 at 5:05 PM, Yu Wei wrote:
> Hi guys,
>
>
> It seemed that when launching application via yarn on single node,
> JavaDStream.print
