hey u can use repartition and set it to 1
as in this example
unionDStream.foreachRDD((rdd, time) => {
val count = rdd.count()
println("count" + count)
if (count > 0) {
print("rdd partition=" + rdd.partitions.length)
val outputRDD =
Hi All,
I am working with Kafka, Spark Streaming and I want to write the
streaming output to a single file. dstream.saveAsTexFiles() is creating
files in different folders. Is there a way to write to a single folder ? or
else if written to different folders, how do I merge them ?
Thanks,
Padma
