Josh Rosen created SPARK-8460: --------------------------------- Summary: Distinguish between shuffle and non-shuffle spills in metrics Key: SPARK-8460 URL: https://issues.apache.org/jira/browse/SPARK-8460 Project: Spark Issue Type: Improvement Components: Spark Core Reporter: Josh Rosen
Some Spark operators may spill to disk even if they are not performing a shuffle (for instance, an external aggregation). We should consider whether we want to distinguish between shuffle- and non-shuffle spill or whether we want to just update the web UI metrics descriptions to make it more clear that the spill may include non-shuffle spills as well. /cc [~kayousterhout] -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org