Bjorn Jonsson created SPARK-27874:
-------------------------------------
Summary: ShuffleBlockFetchIterator can log information about
per-block sizes
Key: SPARK-27874
URL: https://issues.apache.org/jira/browse/SPARK-27874
Project: Spark
Issue Type: Improvement
Components: Shuffle, Spark Core
Affects Versions: 2.4.2
Reporter: Bjorn Jonsson
It can be beneficial for monitoring and debugging to know the size of the
largest shuffle block (or some distribution of block sizes). The
ShuffleBlockFetcherIterator currently logs the number of local & remote blocks
and their total size (see SPARK-26697), but does not log any information about
per-block sizes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
