Here's how you do it:
val joined_stream = *myStream*.transform((x: RDD[(String, String)]) =
{ val prdd = new PairRDDFunctions[String, String](x)
prdd.join(*myRDD*)})
Thanks
Best Regards
On Thu, Jan 8, 2015 at 10:20 PM, Asim Jalis asimja...@gmail.com wrote:
Is there a way
Is there a way to join non-DStream RDDs with DStream RDDs?
Here is the use case. I have a lookup table stored in HDFS that I want to
read as an RDD. Then I want to join it with the RDDs that are coming in
through the DStream. How can I do this?
Thanks.
Asim
You are looking for dstream.transform(rdd = rdd.op(otherRdd))
The docs contain an example on how to use transform.
https://spark.apache.org/docs/latest/streaming-programming-guide.html#transformations-on-dstreams
-kr, Gerard.
On Thu, Jan 8, 2015 at 5:50 PM, Asim Jalis asimja...@gmail.com
