Faiz Halde created SPARK-43408: ---------------------------------- Summary: Spark caching in the context of a single job Key: SPARK-43408 URL: https://issues.apache.org/jira/browse/SPARK-43408 Project: Spark Issue Type: Question Components: Shuffle Affects Versions: 3.3.1 Reporter: Faiz Halde
Does caching benefit a spark job with only a single action in it? Spark IIRC already optimizes shuffles by persisting them onto the disk I am unable to find a counter-example where caching would benefit a job with a single action. In every case I can think of, the shuffle checkpoint acts as a good enough caching mechanism in itself -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org