[
https://issues.apache.org/jira/browse/HDDS-16298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Meng-Shuan Tsai updated HDDS-16298:
-----------------------------------
Description:
Problem
This is a bug in the Ozone client. EC file checksums computed with
ChecksumCombineMode.MD5MD5CRC are silently wrong: every EC block gets the MD5
of empty input (d41d8cd98f00b204e9800998ecf8427e) instead of the MD5 of the
stripe checksums. The default combine mode COMPOSITE_CRC is not affected.
Root cause
ECBlockChecksumComputer.computeMd5Crc calls MessageDigest.digest() twice:
{code:java}
final byte[] fileMD5 = digester.digest();
setOutBytes(digester.digest());{code}
MessageDigest.digest() resets the digest after the first call. The second call
hashes empty input. The DEBUG log prints the first (correct) fileMD5, so the
intended value is discarded.
This was introduced in HDDS-10480 when computeMd5Crc was rewritten to use
MessageDigest.update(). The new code calls digester.digest() twice; the second
call (after reset) is passed to setOutBytes.
was:
Problem
This is a bug in the Ozone client. EC file checksums computed with
ChecksumCombineMode.MD5MD5CRC are silently wrong: every EC block gets the MD5
of empty input (d41d8cd98f00b204e9800998ecf8427e) instead of the MD5 of the
stripe checksums. The default combine mode COMPOSITE_CRC is not affected.
Root cause
ECBlockChecksumComputer.computeMd5Crc calls MessageDigest.digest() twice:
{code:java}
final byte[] fileMD5 = digester.digest();
setOutBytes(digester.digest());{code}
MessageDigest.digest() resets the digest after the first call. The second call
hashes empty input. The DEBUG log prints the first (correct) fileMD5, so the
intended value is discarded.
This was introduced in HDDS-10480 when computeMd5Crc was rewritten from a
single MD5Hash.digest() to MessageDigest.update() + digest().
> ECBlockChecksumComputer.computeMd5Crc calls MessageDigest.digest() twice and
> returns empty MD5
> ----------------------------------------------------------------------------------------------
>
> Key: HDDS-16298
> URL: https://issues.apache.org/jira/browse/HDDS-16298
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Meng-Shuan Tsai
> Assignee: Meng-Shuan Tsai
> Priority: Major
>
> Problem
> This is a bug in the Ozone client. EC file checksums computed with
> ChecksumCombineMode.MD5MD5CRC are silently wrong: every EC block gets the MD5
> of empty input (d41d8cd98f00b204e9800998ecf8427e) instead of the MD5 of the
> stripe checksums. The default combine mode COMPOSITE_CRC is not affected.
> Root cause
> ECBlockChecksumComputer.computeMd5Crc calls MessageDigest.digest() twice:
> {code:java}
> final byte[] fileMD5 = digester.digest();
> setOutBytes(digester.digest());{code}
> MessageDigest.digest() resets the digest after the first call. The second
> call hashes empty input. The DEBUG log prints the first (correct) fileMD5, so
> the intended value is discarded.
> This was introduced in HDDS-10480 when computeMd5Crc was rewritten to use
> MessageDigest.update(). The new code calls digester.digest() twice; the
> second call (after reset) is passed to setOutBytes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
