wuyi created SPARK-36530:
----------------------------
Summary: Avoid finalizing when there's no push at all in a shuffle
Key: SPARK-36530
URL: https://issues.apache.org/jira/browse/SPARK-36530
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 3.2.0, 3.3.0
Reporter: wuyi
When all partition data of a map output is bigger than
spark.shuffle.push.maxBlockSizeToPush, there will be no push. When all map
outputs don't have push, the shuffle doesn't have push. In that case, we don't
need to launch the finalizing request.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
