Access DStream content?

2014-06-12 Thread Wolfinger, Fred 
Good morning.

I have a question related to Spark Streaming. I have reduced some data down
to a simple count value (by window), and would like to take immediate action
on that value before storing in HDFS. I don't see any DStream member
functions that would allow me to access its contents. Is what I am trying to
do not in the spirit of Spark Streaming? If it's not, what would be the best
practice for doing so?

Thanks so much!

Fred





smime.p7s
Description: S/MIME cryptographic signature

Re: Access DStream content?

2014-06-12 Thread Gianluca Privitera 
You can use ForeachRDD then access RDD data.
Hope this works for you.

Gianluca

On 12 Jun 2014, at 10:06, Wolfinger, Fred 
mailto:fwolfin...@cyberpointllc.com>> wrote:

Good morning.

I have a question related to Spark Streaming. I have reduced some data down to 
a simple count value (by window), and would like to take immediate action on 
that value before storing in HDFS. I don't see any DStream member functions 
that would allow me to access its contents. Is what I am trying to do not in 
the spirit of Spark Streaming? If it's not, what would be the best practice for 
doing so?

Thanks so much!

Fred