[ https://issues.apache.org/jira/browse/HDFS-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250369#comment-13250369 ]
Todd Lipcon commented on HDFS-3222: ----------------------------------- Sounds good to me. > DFSInputStream#openInfo should not silently get the length as 0 when > locations length is zero for last partial block. > --------------------------------------------------------------------------------------------------------------------- > > Key: HDFS-3222 > URL: https://issues.apache.org/jira/browse/HDFS-3222 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs client > Affects Versions: 1.0.3, 2.0.0, 3.0.0 > Reporter: Uma Maheswara Rao G > Assignee: Uma Maheswara Rao G > Attachments: HDFS-3222-Test.patch > > > I have seen one situation with Hbase cluster. > Scenario is as follows: > 1)1.5 blocks has been written and synced. > 2)Suddenly cluster has been restarted. > Reader opened the file and trying to get the length., By this time partial > block contained DNs are not reported to NN. So, locations for this partial > block would be 0. In this case, DFSInputStream assumes that, 1 block size as > final size. > But reader also assuming that, 1 block size is the final length and setting > his end marker. Finally reader ending up reading only partial data. Due to > this, HMaster could not replay the complete edits. > Actually this happend with 20 version. Looking at the code, same should > present in trunk as well. > {code} > int replicaNotFoundCount = locatedblock.getLocations().length; > > for(DatanodeInfo datanode : locatedblock.getLocations()) { > .......... > .......... > // Namenode told us about these locations, but none know about the replica > // means that we hit the race between pipeline creation start and end. > // we require all 3 because some other exception could have happened > // on a DN that has it. we want to report that error > if (replicaNotFoundCount == 0) { > return 0; > } > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira