Jungtaek Lim created SPARK-29301: ------------------------------------ Summary: Removing block is not reflected to the driver/executor's storage memory Key: SPARK-29301 URL: https://issues.apache.org/jira/browse/SPARK-29301 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 2.4.4, 3.0.0 Reporter: Jungtaek Lim
While investigating SPARK-29055 I've found that heap memory in driver doesn't increase which would denote memory leak happen, but storage memory doesn't decrease even there're some broadcast blocks getting removed - these values keep increasing. Please refer SPARK-29055 to see the step to reproduce. Very easy to see it, as running simple query repeatedly would reproduce the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org