[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15705840#comment-15705840 ]
Wei-Chiu Chuang commented on HDFS-11160: ---------------------------------------- I have been also trying to optimize checksum calculation. The basic idea is to copy the in-memory last partial chunk checksum in RBW replica when converting an RBW to Finalized, so that there's no need to recalculate checksum every time BlockSender reads a Finalized block. However, maintaining update-to-date last partial chunk checksum in-memory is pretty complex, due to operations such as truncate, as well as use cases such as HSM. May I suggest that we get the current patch reviewed and committed, because it's quite critical, and then work on optimization later? Thanks. > VolumeScanner reports write-in-progress replicas as corrupt incorrectly > ----------------------------------------------------------------------- > > Key: HDFS-11160 > URL: https://issues.apache.org/jira/browse/HDFS-11160 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode > Environment: CDH5.7.4 > Reporter: Wei-Chiu Chuang > Assignee: Wei-Chiu Chuang > Attachments: HDFS-11160.001.patch, HDFS-11160.002.patch, > HDFS-11160.reproduce.patch > > > Due to a race condition initially reported in HDFS-6804, VolumeScanner may > erroneously detect good replicas as corrupt. This is serious because in some > cases it results in data loss if all replicas are declared corrupt. > We are investigating an incidence that caused very high block corruption rate > in a relatively small cluster. Initially, we thought HDFS-11056 is to blame. > However, after applying HDFS-11056, we are still seeing VolumeScanner > reporting corrupt replicas. > It turns out that if a replica is being appended while VolumeScanner is > scanning it, VolumeScanner may use the new checksum to compare against old > data, causing checksum mismatch. > I have a unit test to reproduce the error. Will attach later. > To fix it, I propose a FinalizedReplica object should also have a > lastChecksum field like ReplicaBeingWritten, and BlockSender should use the > in-memory lastChecksum to verify the partial data in the last chunk on disk. > File this jira to discuss a good fix for this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org