Hi all:
We implemented an balanced block placement policy in HDFS-8131.
https://issues.apache.org/jira/browse/HDFS-8131
Could someone help to review and push this patch?
This policy will choose the low used percent datanodes for new blocks
with a little high possibility.
In a not long term, the used percent of datanodes will trend to be balanced.
If there are p percent new added nodes in the cluster.
In the default block placement policy, the new added nodes will be
chosen with possibility: p.
In the balanced block placement policy in this patch, the possibility
of choosing new added nodes will be q = p * p + 2* (1-p) * p * 0.6.
(0.6 is configurable)
You can see the code from the patch.
if p = 0.5, q = 0.55. if p = 0.01, q = 0.01198.
The percent of increased possibility is not very exceeding。
The issue last for a month and we want push it forward, so I send a this
mail to dev mail list.
Sorry for disappointing you.
-Shaohui Liu from Xiaomi
