[
https://issues.apache.org/jira/browse/HADOOP-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated HADOOP-659:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Hairong!
> Boost the priority of re-replicating blocks that are far from their
> replication target
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-659
> URL: https://issues.apache.org/jira/browse/HADOOP-659
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.10.1
> Reporter: Konstantin Shvachko
> Assigned To: Hairong Kuang
> Fix For: 0.11.0
>
> Attachments: priBlockRep2.patch
>
>
> I see two types of replications that should be accelerated compared to all
> others.
> 1. Blocks that have only one remaining copy (but are required to have higher
> replication).
> 2. Blocks that have less than 1/3 of their replicas in place.
> The latter occurs when map/reduce sets replication of certain files to 10,
> and we want
> it happen fast to achieve better performance on the tasks.
> So I think we should distinguish two major groups of under-replicated blocks:
> first-priority (having only 1 copy or less than 1/3 of required replicas),
> and the rest.
> The name-node places first-priority blocks into the beginning of the
> neededReplication
> list, and the rest are placed at the end. That way the first-priority blocks
> will be replicated
> first and then the others.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
