http://docs.sigmoidanalytics.com/index.php/Checkpoint_and_not_running_out_of_disk_space
On Mon, Apr 14, 2014 at 2:43 AM, Cheng Lian <lian.cs....@gmail.com> wrote: > Checkpointed RDDs are materialized on disk, while cached RDDs are > materialized in memory. When memory is insufficient, cached RDD blocks (1 > block per partition) will be evicted in an LRU manner. An evicted RDD block > will be spilled to disk if the storage level of the RDD allows, otherwise > this block vanishes entirely and must be recomputed from the lineage DAG if > it's referenced later. > > > On Mon, Apr 14, 2014 at 10:20 AM, David Thomas <dt5434...@gmail.com>wrote: > >> What is the difference between checkpointing and caching an RDD? >> > >
