Please take a look
at
streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala
def saveAsHadoopFiles[F <: OutputFormat[K, V]](
prefix: String,
suffix: String
)(implicit fm: ClassTag[F]): Unit = ssc.withScope {
Cheers
On Sat, Jul 4, 2015 at 5:23 PM
Hello,
How should I write a text file stream DStream to HDFS.
I tried the the following
val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input/")
lines.saveAsTextFile("hdfs:/user/hadoop/output1")
val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input/")
