[ http://issues.apache.org/jira/browse/HADOOP-659?page=comments#action_12446507 ] eric baldeschwieler commented on HADOOP-659: --------------------------------------------
why not maintain a total order? > Boost the priority of re-replicating blocks that are far from their > replication target > -------------------------------------------------------------------------------------- > > Key: HADOOP-659 > URL: http://issues.apache.org/jira/browse/HADOOP-659 > Project: Hadoop > Issue Type: Improvement > Components: dfs > Affects Versions: 0.7.2 > Reporter: Konstantin Shvachko > Assigned To: Konstantin Shvachko > > I see two types of replications that should be accelerated compared to all > others. > 1. Blocks that have only one remaining copy (but are required to have higher > replication). > 2. Blocks that have less than 1/3 of their replicas in place. > The latter occurs when map/reduce sets replication of certain files to 10, > and we want > it happen fast to achieve better performance on the tasks. > So I think we should distinguish two major groups of under-replicated blocks: > first-priority (having only 1 copy or less than 1/3 of required replicas), > and the rest. > The name-node places first-priority blocks into the beginning of the > neededReplication > list, and the rest are placed at the end. That way the first-priority blocks > will be replicated > first and then the others. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
