Hi Maria, If you don't have a reference to a persisted RDD, it will be automatically unpersisted on the next GC by the ContextCleaner. This is implemented for scala, but should still work in python because python uses reference counting to clean up objects that are no longer strongly referenced.
Andrew 2014-07-21 8:37 GMT-07:00 mrm <ma...@skimlinks.com>: > Hi, > > I am using pyspark and have persisted a list of rdds within a function, but > I don't have a reference to them anymore. The RDD's are listed in the UI, > under the "Storage" tab, and they have names associated to them (e.g. 4). > Is > it possible to access the RDD's to unpersist them? > > Thanks! > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/gain-access-to-persisted-rdd-tp10313.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
