http://docs.sigmoidanalytics.com/index.php/Checkpoint_and_not_running_out_of_disk_space
On Mon, Apr 14, 2014 at 2:43 AM, Cheng Lian <[email protected]> wrote: > Checkpointed RDDs are materialized on disk, while cached RDDs are > materialized in memory. When memory is insufficient, cached RDD blocks (1 > block per partition) will be evicted in an LRU manner. An evicted RDD block > will be spilled to disk if the storage level of the RDD allows, otherwise > this block vanishes entirely and must be recomputed from the lineage DAG if > it's referenced later. > > > On Mon, Apr 14, 2014 at 10:20 AM, David Thomas <[email protected]>wrote: > >> What is the difference between checkpointing and caching an RDD? >> > >
