[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008665#comment-13008665 ]
dhruba borthakur commented on HDFS-1172: ---------------------------------------- putting it in pendingReplication means that replication (when needed) will occur only after 5 minutes. This is a long time, isn't it? Maybe it is better to put it in neededReplication but (somehow) ensure that it is not attempted to be replicated until after a small delay. > Blocks in newly completed files are considered under-replicated too quickly > --------------------------------------------------------------------------- > > Key: HDFS-1172 > URL: https://issues.apache.org/jira/browse/HDFS-1172 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node > Affects Versions: 0.21.0 > Reporter: Todd Lipcon > Attachments: HDFS-1172.patch > > > I've seen this for a long time, and imagine it's a known issue, but couldn't > find an existing JIRA. It often happens that we see the NN schedule > replication on the last block of files very quickly after they're completed, > before the other DNs in the pipeline have a chance to report the new block. > This results in a lot of extra replication work on the cluster, as we > replicate the block and then end up with multiple excess replicas which are > very quickly deleted. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira