Hello team, Spark version: 1.6.0
I'm trying to persist done data into memory for reusing them. However, when I call rdd.cache() OR rdd.persist(StorageLevel.MEMORY_ONLY()) it does not store the data as I can not see any rdd information under WebUI (Storage Tab). Therefore I tried rdd.persist(StorageLevel.MEMORY_AND_DISK()), for which it stored the data into Disk only as shown in below screenshot: [image: Inline images 2] Do you know why the memory is not being used? Is there a configuration in cluster level to stop jobs from storing data into memory altogether? Please let me know. Thanks Guru
