[ https://issues.apache.org/jira/browse/HDFS-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay updated HDFS-5177: ------------------------ Assignee: Vinay Status: Patch Available (was: Open) > blocksScheduled count should be decremented for abandoned blocks > ----------------------------------------------------------------- > > Key: HDFS-5177 > URL: https://issues.apache.org/jira/browse/HDFS-5177 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.1.0-beta, 3.0.0 > Reporter: Vinay > Assignee: Vinay > Attachments: HDFS-5177.patch > > > DatanodeDescriptor#incBlocksScheduled() will be called for all datanodes of > the block on each allocation. But same should be decremented for abandoned > blocks. > When one of the datanodes is down and same is allocated for the block along > with other live datanodes, then this block will be abandoned, but the > scheduled count on other datanodes will consider live datanodes as loaded, > but in reality these datanodes may not be loaded. > Anyway this scheduled count will be rolled every 20 mins. > Problem will come if the rate of creation of files is more. Due to increase > in the scheduled count, there might be chances of missing local datanode to > write to. and some times writes also can fail in small clusters. > So we need to decrement the unnecessary count on abandon block call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira