i noticed that some algorithms such as graphx liberally cache RDDs for efficiency, which makes sense. however it can also leave a long trail of unused yet cached RDDs, that might push other RDDs out of memory.
in a long-lived spark context i would like to decide which RDDs stick around. would it make sense to create tiers of caching, to distinguish explicitly cached RDDs by the application from RDDs that are temporary cached by algos, so as to make sure these temporary caches don't push application RDDs out of memory?
