[
https://issues.apache.org/jira/browse/HDFS-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14552602#comment-14552602
]
Hudson commented on HDFS-8131:
------------------------------
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2149 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2149/])
HDFS-8131. Implement a space balanced block placement policy. Contributed by
Liu Shaohui. (kihwal: rev de30d66b2673d0344346fb985e786247ca682317)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestAvailableSpaceBlockPlacementPolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
> Implement a space balanced block placement policy
> -------------------------------------------------
>
> Key: HDFS-8131
> URL: https://issues.apache.org/jira/browse/HDFS-8131
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Minor
> Labels: BlockPlacementPolicy
> Fix For: 2.8.0
>
> Attachments: HDFS-8131-v1.diff, HDFS-8131-v2.diff, HDFS-8131-v3.diff,
> HDFS-8131.004.patch, HDFS-8131.005.patch, HDFS-8131.006.patch, balanced.png
>
>
> The default block placement policy will choose datanodes for new blocks
> randomly, which will result in unbalanced space used percent among datanodes
> after an cluster expansion. The old datanodes always are in high used percent
> of space and new added ones are in low percent.
> Through we can used the external balance tool to balance the space used rate,
> it will cost extra network IO and it's not easy to control the balance speed.
> An easy solution is to implement an balanced block placement policy which
> will choose low used percent datanodes for new blocks with a little high
> possibility. In a not long term, the used percent of datanodes will trend to
> be balanced.
> Suggestions and discussions are welcomed. Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
