Re: Spark streaming question - SPARK-13758 Need to use an external RDD inside DStream processing...Please help

You can create lazily instantiated singleton instances. See http://spark.apache.org/docs/latest/streaming-programming-guide.html#accumulators-broadcast-variables-and-checkpoints for examples of accumulators and broadcast variables. You can use the same approach to create your cached RDD. On Tue,

Re: Spark streaming question - SPARK-13758 Need to use an external RDD inside DStream processing...Please help

and my cached RDD is not small. If it was maybe I could materialize and broadcast. Thanks On Tue, Feb 7, 2017 at 10:28 AM, shyla deshpande wrote: > I have a situation similar to the following and I get SPARK-13758 >

Spark streaming question - SPARK-13758 Need to use an external RDD inside DStream processing...Please help

I have a situation similar to the following and I get SPARK-13758 . I understand why I get this error, but I want to know what should be the approach in dealing with these situations. Thanks > var cached =