jin xing created SPARK-19937: -------------------------------- Summary: Collect metrics of block sizes when shuffle. Key: SPARK-19937 URL: https://issues.apache.org/jira/browse/SPARK-19937 Project: Spark Issue Type: Improvement Components: Shuffle Affects Versions: 2.1.0 Reporter: jin xing
This improvement is related to SPARK-19659 and also documented there. Metrics of blocks sizes(when shuffle) should be collected for later analysis. This is helpful for analysis when skew situations or OOM happens(though maxBytesInFlight is set). -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org