RexXiong commented on code in PR #2979:
URL: https://github.com/apache/celeborn/pull/2979#discussion_r1883515658
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala:
##########
@@ -1470,7 +1470,24 @@ class PushDataHandler(val workerSource: WorkerSource)
extends BaseMessageHandler
shuffleKey: String,
index: Int): Unit = {
try {
- fileWriter.write(body)
+ val header = new
Array[Byte](PushDataHeaderUtils.BATCH_HEADER_SIZE_WITHOUT_CHECKSUM)
Review Comment:
We could consider implementing a new method for verifying data checksums. If
a checksum validation fails, we would throw a CelebornChecksumException. This
approach would easier for ut testing. Additionally, we can handle this
exception within the catch block alongside other exceptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
