Hi Q1: Is .checkpoint() the same as .cache() but the difference is that it writes to disk instead of memory?
Q2: When is the checkpointed data rewritten in a streaming context? I assume it is rewritten otherwise the file where the stream is checkpointed to would grow forever. Is it every batch_duration? ie: For val streamingContext = new StreamingContext(conf, Seconds(10)) the checkpointed data would be rewritten every 10 seconds. Thanks -Adrian
