[
https://issues.apache.org/jira/browse/HDFS-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762818#action_12762818
]
Hairong Kuang commented on HDFS-679:
------------------------------------
The problem is caused by incorrect handling of checksum when appending to a
partial chunk. Specifically the following code in
DFSClient#DFSOutputStream#writeChunk:
{noformat}
// If this was the first write after reopening a file, then the above
// write filled up any partial chunk. Tell the summer to generate full
// crc chunks from now on.
if (appendChunk) {
appendChunk = false;
resetChecksumChunk(bytesPerChecksum);
}
{noformat}
The code blindly assumes that the first packet after reopening a file for
append fills up the partial chunk. But in case of hflush, a packet may be sent
before the partial chunk is filled up. So resetting the checksum calculator
makes the next packet to be sent with a wrong checkusm.
> Block receiver unexpectedly throws an IOException complaining mismatched
> checksum
> ---------------------------------------------------------------------------------
>
> Key: HDFS-679
> URL: https://issues.apache.org/jira/browse/HDFS-679
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.21.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.21.0
>
>
> When I run TestClientProtocolForPipelineRecovery, I always see that the block
> receiver throws IOException complaining about mismatched checksum when
> receiving the last data packet. It turns out the checksum of last packet was
> unexpectedly set to be zero.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
