yandrey321 commented on code in PR #10764:
URL: https://github.com/apache/ozone/pull/10764#discussion_r3763148114
##########
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java:
##########
@@ -1162,4 +1163,106 @@ boolean deleteUnreferencedFile(File file) {
return false;
}
}
+
+ @Test
+ public void testGetChecksumsWithVaryingChunkSizes() {
+ int bytesPerChecksum = 1024;
+ int bytesPerChunk = 16 * 1024;
+
+ ContainerProtos.ChunkInfo chunk1 = ContainerProtos.ChunkInfo.newBuilder()
+ .setChunkName("chunk1")
+ .setOffset(0)
+ .setLen(1024)
+ .setChecksumData(ContainerProtos.ChecksumData.newBuilder()
+ .setType(ContainerProtos.ChecksumType.CRC32)
+ .setBytesPerChecksum(bytesPerChecksum)
+
.addChecksums(org.apache.ratis.thirdparty.com.google.protobuf.ByteString.copyFromUtf8("chk1"))
+ .build())
+ .build();
+
+ ContainerProtos.ChunkInfo chunk2 = ContainerProtos.ChunkInfo.newBuilder()
+ .setChunkName("chunk2")
+ .setOffset(1024)
+ .setLen(10)
+ .setChecksumData(ContainerProtos.ChecksumData.newBuilder()
+ .setType(ContainerProtos.ChecksumType.CRC32)
+ .setBytesPerChecksum(bytesPerChecksum)
+
.addChecksums(org.apache.ratis.thirdparty.com.google.protobuf.ByteString.copyFromUtf8("chk2"))
+ .build())
+ .build();
+
+ ContainerProtos.ChunkInfo chunk3 = ContainerProtos.ChunkInfo.newBuilder()
+ .setChunkName("chunk3")
+ .setOffset(1034)
+ .setLen(2048)
+ .setChecksumData(ContainerProtos.ChecksumData.newBuilder()
+ .setType(ContainerProtos.ChecksumType.CRC32)
+ .setBytesPerChecksum(bytesPerChecksum)
+
.addChecksums(org.apache.ratis.thirdparty.com.google.protobuf.ByteString.copyFromUtf8("chk3-1"))
+
.addChecksums(org.apache.ratis.thirdparty.com.google.protobuf.ByteString.copyFromUtf8("chk3-2"))
+ .build())
+ .build();
+
+ List<ContainerProtos.ChunkInfo> chunks = java.util.Arrays.asList(chunk1,
chunk2, chunk3);
+
+ // Read full block (1024 + 10 + 2048)
+ List<ByteString> checksums = KeyValueHandler.getChecksums(0, 3082,
bytesPerChunk, bytesPerChecksum, chunks);
+ assertEquals(4, checksums.size());
+ assertEquals("chk1", checksums.get(0).toStringUtf8());
+ assertEquals("chk2", checksums.get(1).toStringUtf8());
+ assertEquals("chk3-1", checksums.get(2).toStringUtf8());
+ assertEquals("chk3-2", checksums.get(3).toStringUtf8());
+
+ // Read from offset 1024
+ checksums = KeyValueHandler.getChecksums(1024, 2058, bytesPerChunk,
bytesPerChecksum, chunks);
+ assertEquals(3, checksums.size());
+ assertEquals("chk2", checksums.get(0).toStringUtf8());
+ assertEquals("chk3-1", checksums.get(1).toStringUtf8());
+ assertEquals("chk3-2", checksums.get(2).toStringUtf8());
+
+ // Read from offset 2048
+ checksums = KeyValueHandler.getChecksums(2048, 1034, bytesPerChunk,
bytesPerChecksum, chunks);
+ assertEquals(2, checksums.size());
+ assertEquals("chk3-1", checksums.get(0).toStringUtf8());
Review Comment:
The read starts at block offset 2048, but chunk3 starts at 1034, so 2048 is
internal offset 2048 - 1034 = 1014 inside chunk3 — not a checksum-window
boundary (1014 % 1024 != 0). chk3-1 is the checksum of chunk3's first window,
covering block bytes [1034, 2058), whereas the returned data begins at 2048. So
checksums.get(0) does not match the first bytes of the read.
The test passes only because the chunks carry placeholder labels ("chk3-1")
instead of real CRCs over real bytes, so nothing ties the checksum to the data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
