[
https://issues.apache.org/jira/browse/SPARK-54830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tengfei Huang updated SPARK-54830:
----------------------------------
Description:
With https://issues.apache.org/jira/browse/SPARK-51756 , we have implemented a
checksum based solution to detect shuffle output changes and fully retry all
the consumer stages to ensure result correctness.
Enable the feature by default for result correctness with indeterminate shuffle
retry.
> Enable checksum based indeterminate shuffle retry by default
> ------------------------------------------------------------
>
> Key: SPARK-54830
> URL: https://issues.apache.org/jira/browse/SPARK-54830
> Project: Spark
> Issue Type: Task
> Components: Spark Core
> Affects Versions: 4.1.0
> Reporter: Tengfei Huang
> Priority: Minor
>
> With https://issues.apache.org/jira/browse/SPARK-51756 , we have implemented
> a checksum based solution to detect shuffle output changes and fully retry
> all the consumer stages to ensure result correctness.
> Enable the feature by default for result correctness with indeterminate
> shuffle retry.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
