[
https://issues.apache.org/jira/browse/HDFS-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763192#action_12763192
]
Hairong Kuang commented on HDFS-679:
------------------------------------
I'd like to propose a solution to the appending to a partial checksum chunk
problem. The client reads in the partial chunk into its buffer before any write
and they will be part of the packets sent to the datanode side before the chunk
gets filled up. This solution maintains a good property: any packet sent to
datanodes starts at a checksum chunk boundary. This would not increase
complexity at the client side but would completely eliminate the handling of
appending to partial chunk at the datanode side.
Another optimization to a block receiver at the datanode is not to overwrite
block file if a packet starts with data that the replica file already has. For
the crc file, only the last 4 bytes are allowed to be overwritten.
> Block receiver unexpectedly throws an IOException complaining mismatched
> checksum
> ---------------------------------------------------------------------------------
>
> Key: HDFS-679
> URL: https://issues.apache.org/jira/browse/HDFS-679
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.21.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.21.0
>
>
> When I run TestClientProtocolForPipelineRecovery, I always see that the block
> receiver throws IOException complaining about mismatched checksum when
> receiving the last data packet. It turns out the checksum of last packet was
> unexpectedly set to be zero.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
