[ https://issues.apache.org/jira/browse/SPARK-32922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17249272#comment-17249272 ]
Apache Spark commented on SPARK-32922: -------------------------------------- User 'otterc' has created a pull request for this issue: https://github.com/apache/spark/pull/30768 > Add support for ShuffleBlockFetcherIterator to read from merged shuffle > partitions and to fallback to original shuffle blocks if encountering failures > ------------------------------------------------------------------------------------------------------------------------------------------------------ > > Key: SPARK-32922 > URL: https://issues.apache.org/jira/browse/SPARK-32922 > Project: Spark > Issue Type: Sub-task > Components: Shuffle, Spark Core > Affects Versions: 3.1.0 > Reporter: Min Shen > Priority: Major > > With the extended MapOutputTracker, the reducers can now get the task input > data from the merged shuffle partitions for more efficient shuffle data > fetch. The reducers should also be able to fallback to fetching the original > unmarked blocks if it encounters failures when fetching the merged shuffle > partitions. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org