Hi, By default a fraction of the executor memory (60%) is reserved for RDD caching, so if there's no explicit caching in the code (eg. rdd.cache() etc.), or if we persist RDD with StorageLevel.DISK_ONLY, is this part of memory wated? Does Spark allocates the RDD cache memory dynamically? Or does spark automatically caches RDDs when it can?
Thanks. -- *JU Han* Data Engineer @ Botify.com +33 0619608888
