smengcl commented on code in PR #10350:
URL: https://github.com/apache/ozone/pull/10350#discussion_r3762705472
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChecksumCache.java:
##########
@@ -65,58 +66,102 @@ public List<ByteString> getChecksums() {
return checksums;
}
- public List<ByteString> computeChecksum(ChunkBuffer data,
Function<ByteBuffer, ByteString> function) {
- // Indicates how much data the current chunk buffer holds
+ /**
+ * Recompute checksums for the windows that have changed since the last
+ * call: bytes {@code [ciStart * bytesPerChecksum, currChunkLength)} where
+ * {@code ciStart = prevChunkLength / bytesPerChecksum} (the index of the
+ * first window whose result may have changed - either the previously-
+ * partial last window now has more bytes, or new full windows have been
+ * appended).
+ *
+ * <p>Walks {@code data}'s underlying buffer list directly (no
+ * {@code iterate()} byte[] linearization) and feeds slices to {@code algo}
+ * incrementally; the cached prefix is skipped via index arithmetic so
+ * those bytes are never re-fed.
+ */
+ List<ByteString> computeChecksum(ChunkBuffer data,
+ StreamingChecksum algo, int chksumSize) {
+ if (chksumSize != bytesPerChecksum) {
+ throw new IllegalArgumentException("bytesPerChecksum mismatch: cache="
+ + bytesPerChecksum + " call=" + chksumSize);
+ }
final int currChunkLength = data.limit();
Review Comment:
Good catch. This is a latent issue though it wasn't changed here.
It doesn't reveal because `data.position()` is guaranteed to be zero with
the current usage in ozone (caller always invokes flip() before the cached
checksum calculation).
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
