Off the top of my head, you can use the ForEachDStream to which you pass
in the code that writes to Hadoop, and then register that as an output
stream, so the function you pass in is periodically executed and causes
the data to be written to HDFS. If you are ok with the data being in
text format - simply use saveAsTextFiles method in the RDD class.
salemi wrote:
Hi,
I was wondering what is the best way to store off dstreams in hdfs or
casandra.
Could somebody provide an example?
Thanks,
Ali
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/store-spark-streaming-dstream-in-hdfs-or-cassandra-tp11064.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
