Hi Yes data would be cached again in each spark context. Regards Mayur
On Friday, August 1, 2014, Sujee Maniyam <su...@sujee.net> wrote: > Hi all, > I have a scenario of a web application submitting multiple jobs to Spark. > These jobs may be operating on the same RDD. > > It is possible to cache() the RDD during one call... > And all subsequent calls can use the cached RDD? > > basically, during one invocation > val rdd1 = sparkContext1.textFile( file1).cache () > > another invocation.. > val rdd2 = sparkContext2.textFile(file1).cache() > > (note that spark context are different, but the file is the same) > > will the same file be loaded again in another spark context? > or there will be only one cached copy (since RDDs are immutable) > > thanks! > Sujee Maniyam (http://sujee.net | http://www.linkedin.com/in/sujeemaniyam > ) > -- Sent from Gmail Mobile
