Tengfei Huang created SPARK-54663:
-------------------------------------
Summary: Computes RowBasedChecksum in ShuffleWriters
Key: SPARK-54663
URL: https://issues.apache.org/jira/browse/SPARK-54663
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 4.0.0
Reporter: Tengfei Huang
To help detect the shuffle output changes due to indeterminate stage retry, we
need to introduce an order independent checksum for the shuffle output.
Once the checksum changes, we'll know that this is from a indeterminate stage
retry, will need to recompute all the succeeding stages for data correctness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
