[ https://issues.apache.org/jira/browse/HDFS-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871285#action_12871285 ]
sam rash commented on HDFS-1057: -------------------------------- @hairong: I'm looking a little at implementing this in trunk (reading your append/hflush doc from hdfs-265), and I have a question. From above: "In each ReplcaBeingWritten, we could have two more fields to keep track of the last consistent state: replica length and the last chunk's crc" why does there need to be another length field? the getVisibleLenght() == acked bytes isn't sufficient? if the crc stored in the RBW is for that length, you only need the additional byte[] field which is the last chunk's crc I think. ReplicaBeingWritten.setBytesAcked() could take the crc and atomically set the len + bytes > Concurrent readers hit ChecksumExceptions if following a writer to very end > of file > ----------------------------------------------------------------------------------- > > Key: HDFS-1057 > URL: https://issues.apache.org/jira/browse/HDFS-1057 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: data-node > Affects Versions: 0.21.0, 0.22.0 > Reporter: Todd Lipcon > Assignee: sam rash > Priority: Blocker > Attachments: conurrent-reader-patch-1.txt, > conurrent-reader-patch-2.txt, conurrent-reader-patch-3.txt > > > In BlockReceiver.receivePacket, it calls replicaInfo.setBytesOnDisk before > calling flush(). Therefore, if there is a concurrent reader, it's possible to > race here - the reader will see the new length while those bytes are still in > the buffers of BlockReceiver. Thus the client will potentially see checksum > errors or EOFs. Additionally, the last checksum chunk of the file is made > accessible to readers even though it is not stable. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.