You can do something like this:
val rddQueue = scala.collection.mutable.Queue(rdd1,rdd2,rdd3) val qDstream = ssc.queueStream(rddQueue) Thanks Best Regards On Sat, Oct 24, 2015 at 4:43 AM, Anfernee Xu <anfernee...@gmail.com> wrote: > Hi, > > Here's my situation, I have some kind of offline dataset, but I want to > form a virtual data stream feeding to Spark Streaming, my code looks like > this > > > // sort offline data by time > 1) JavaRDD sortedByTime = offlineDataRDD.sortBy( ); > > // compute a list of JavaRDD, each element JavaRDD is hosting the data > in the same time > // bucket. > 2) List<JavaRDD> virtualStreamRdd = ? > > Queue<JavaRDD<Row>> queue = Queues.newLinkedBlockingQueue(); > queue.addAll(virtualStreamRdd); > > /* > * Create DStream from the queue > */ > > 3) final JavaDStream<Row> rowDStream = > streamingContext.queueStream(queue); > > > Currently I'm stucking in 2), any suggestion is appreciated. > > Thanks > > -- > --Anfernee >
