Bjorn Jonsson created SPARK-27874: ------------------------------------- Summary: ShuffleBlockFetchIterator can log information about per-block sizes Key: SPARK-27874 URL: https://issues.apache.org/jira/browse/SPARK-27874 Project: Spark Issue Type: Improvement Components: Shuffle, Spark Core Affects Versions: 2.4.2 Reporter: Bjorn Jonsson
It can be beneficial for monitoring and debugging to know the size of the largest shuffle block (or some distribution of block sizes). The ShuffleBlockFetcherIterator currently logs the number of local & remote blocks and their total size (see SPARK-26697), but does not log any information about per-block sizes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org