[ https://issues.apache.org/jira/browse/HDFS-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043146#comment-13043146 ]
Daryn Sharp commented on HDFS-1907: ----------------------------------- I might very well be misreading the code... It looks like the only way to leave {{getFinalizedBlockRange}} is either: * Locate all blocks for the range spanned by (offset, length) * Exceed the file bounds and fail on {code}assert curOff >= blk.getStartOffset() : "Block not found";{code} Thus when all blocks are finalized, reading past EOF appears to trigger an assertion. However, when the last block is unfinalized, the behavior appears to be different/inconsistent: # {{getFinalizedBlockRange}} is called with a clipped length -- perhaps specifically to avoid the assertion? # Retrieves the unfinalized block. # TODO? Assert if the read request exceeds the length of the unfinalized + finalized blocks. > BlockMissingException upon concurrent read and write: reader was doing file > position read while writer is doing write without hflush > ------------------------------------------------------------------------------------------------------------------------------------ > > Key: HDFS-1907 > URL: https://issues.apache.org/jira/browse/HDFS-1907 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs client > Affects Versions: 0.23.0 > Environment: Run on a real cluster. Using the latest 0.23 build. > Reporter: CW Chung > Assignee: John George > Attachments: HDFS-1907-2.patch, HDFS-1907-3.patch, HDFS-1907-4.patch, > HDFS-1907-5.patch, HDFS-1907-5.patch, HDFS-1907.patch > > > BlockMissingException is thrown under this test scenario: > Two different processes doing concurrent file r/w: one read and the other > write on the same file > - writer keep doing file write > - reader doing position file read from beginning of the file to the visible > end of file, repeatedly > The reader is basically doing: > byteRead = in.read(currentPosition, buffer, 0, byteToReadThisRound); > where CurrentPostion=0, buffer is a byte array buffer, byteToReadThisRound = > 1024*10000; > Usually it does not fail right away. I have to read, close file, re-open the > same file a few times to create the problem. I'll pose a test program to > repro this problem after I've cleaned up a bit my current test program. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira