JavaPairRDD should have had a unpersist() method; we'll fix this bug in 0.8.1 (see https://github.com/apache/incubator-spark/pull/103). In the meantime, just call myJavaPairRDD.rdd().unpersist() (see https://mail-archives.apache.org/mod_mbox/incubator-spark-user/201310.mbox/%3ccaoepxp5syqn7v9gdomjcvgqzvaad-epewxi025jpmu_neic...@mail.gmail.com%3E ).
On Wed, Nov 20, 2013 at 4:39 PM, sasmita Patra <sasmita.pa...@gmail.com>wrote: > Hi, > I have two datasets that i load from HDFS file system. After loading the > files, i cache the datasets. > > I have a requirement to do join on these datasets (LEFT/RIGHT INNER/OUTER > JOIN) and apply some filter conditions and then run multiple queries on > this joined filtered dataset. > > I have created JavaPairRDDs for the datasets, doing the join, applying > filter and caching the dataset. > > After running the queries, i need to unpersist this dataset. Currently i > don't see any unpersist api for JavaPairRDD. > > Is there any way to unpersist the JavaPairRDD? > > Thanks, > Sasmita > >
