[ 
https://issues.apache.org/jira/browse/HDFS-10530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

GAO Rui updated HDFS-10530:
---------------------------
    Description: 
This issue was found by [~tfukudom].

Under RS-DEFAULT-6-3-64k EC policy, 
1. Create an EC file, the file was witten to all the 5 racks( 2 dns for each) 
of the cluster.
2. Reconstruction work would be scheduled if the 6th rack is added. 
3. While adding the 7th rack or more racks will not trigger reconstruction 
work. 

Based on default EC block placement policy defined in 
“BlockPlacementPolicyRackFaultTolerant.java”, EC file should be able to be 
scheduled to distribute to 9 racks if possible.

In *BlockManager#isPlacementPolicySatisfied(BlockInfo storedBlock)* , 
*numReplicas* of striped blocks might should be *getRealTotalBlockNum()*, 
instead of *getRealDataBlockNum()*.

  was:
Under RS-DEFAULT-6-3-64k EC policy, 
This issue was found by [~tfukudom].
1. Create an EC file, the file was witten to all the 5 racks( 2 dns for each) 
of the cluster.
2. Reconstruction work would be scheduled if the 6th rack is added. 
3. While adding the 7th rack or more racks will not trigger reconstruction 
work. 

Based on default EC block placement policy defined in 
“BlockPlacementPolicyRackFaultTolerant.java”, EC file should be able to be 
scheduled to distribute to 9 racks if possible.

In *BlockManager#isPlacementPolicySatisfied(BlockInfo storedBlock)* , 
*numReplicas* of striped blocks might should be *getRealTotalBlockNum()*, 
instead of *getRealDataBlockNum()*.


> BlockManager reconstruction work scheduling should correctly adhere to EC 
> block placement policy
> ------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-10530
>                 URL: https://issues.apache.org/jira/browse/HDFS-10530
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: GAO Rui
>            Assignee: GAO Rui
>
> This issue was found by [~tfukudom].
> Under RS-DEFAULT-6-3-64k EC policy, 
> 1. Create an EC file, the file was witten to all the 5 racks( 2 dns for each) 
> of the cluster.
> 2. Reconstruction work would be scheduled if the 6th rack is added. 
> 3. While adding the 7th rack or more racks will not trigger reconstruction 
> work. 
> Based on default EC block placement policy defined in 
> “BlockPlacementPolicyRackFaultTolerant.java”, EC file should be able to be 
> scheduled to distribute to 9 racks if possible.
> In *BlockManager#isPlacementPolicySatisfied(BlockInfo storedBlock)* , 
> *numReplicas* of striped blocks might should be *getRealTotalBlockNum()*, 
> instead of *getRealDataBlockNum()*.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to