[ https://issues.apache.org/jira/browse/MAPREDUCE-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886816#action_12886816 ]
Scott Chen commented on MAPREDUCE-1861: --------------------------------------- Thanks for the review, Rodrigo. I am resubmitting this to Hudson because there has been no response for a while. > Raid should rearrange the replicas while raiding > ------------------------------------------------ > > Key: MAPREDUCE-1861 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1861 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid > Affects Versions: 0.22.0 > Reporter: Scott Chen > Assignee: Scott Chen > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1861.txt > > > Raided file introduce extra dependencies on the blocks on the same stripe. > Therefore we need a new way to place the blocks. > It is desirable that raided file satisfies the following two conditions: > a. Replicas on the same stripe should be on different machines (or racks) > b. Replicas of the same block should be on different racks > MAPREDUCE-1831 will try to delete the replicas on the same stripe and the > same machine (a). > But in the mean time, it will try to maintain the number of distinct racks of > one block (b). > We cannot satisfy (a) and (b) at the same time with the current logic in > BlockPlacementPolicyDefault.chooseTarget(). > One choice we have is to change BlockPlacementPolicyDefault.chooseTarget(). > However, this placement is in general good for all files including the > unraided ones. > It is not clear to us that we can make this good for both raided and unraided > files. > So we propose this idea that when raiding the file. We create one more > off-rack replica (so the replication=4 now). > Than we delete two blocks using the policy in MAPREDUCE-1831 after that > (replication=2 now). > This way we can rearrange the replicas to satisfy (a) and (b) at the same > time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.