[
https://issues.apache.org/jira/browse/HDFS-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2821:
-----------------------------------------
Assignee: Devaraj K
> Improve the Balancer to move data from over utilized nodes to under utilized
> nodes using balanced nodes
> -------------------------------------------------------------------------------------------------------
>
> Key: HDFS-2821
> URL: https://issues.apache.org/jira/browse/HDFS-2821
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 0.20.205.0, 0.24.0, 0.23.1
> Reporter: Devaraj K
> Assignee: Devaraj K
>
> h5.Cluster State Before Balancer Run:
> ||Node||Last Contact||Admin
> State||Configured||Capacity(TB)||Used(TB)||Remaining(TB)||Used(%)||Remaining(%)||Blocks||
> |xxx-x-xx-n1|0|In Service|4.25|1.76| 0.84|1.65|41.34|38.86|8465|
> |xxx-x-xx-n2|1|In Service|6.03|1.76|0.94 |3.33|29.1|55.24|8465|
> |xxx-x-xx-n3|2|In Service|6.93|1.76|0.99 |4.18|25.35|60.31|8465|
> |xxx-x-xx-n4|2|In Service|10.5|0|0.54|9.97|0|94.9|0|
> \\
> \\
> h5.Cluster State After Balancer Run:
> ||Node||Last Contact||Admin
> State||Configured||Capacity(TB)||Used(TB)||Remaining(TB)||Used(%)||Remaining(%)||Blocks||
> |xxx-x-xx-n1|2|In Service|4.25|0.95|0.84|2.46|22.36|57.84|4830|
> |xxx-x-xx-n2|1|In Service|6.03|1.2|0.94|3.88|19.95|64.4|5858|
> |xxx-x-xx-n3|0|In Service|6.93|1.38|0.99|4.56|19.9|65.76|6327|
> |xxx-x-xx-n4|2|In Service|10.5|1.74|0.54|8.23|16.53|78.37|8383|
> \\
> Currently balancer moves the data from over utilized nodes to the under
> utilized nodes and this process continues till the cluster balanced or there
> is no data to move from source to destination. In this process if some nodes
> usage comes to avgUtilization these will not be participated in the balance
> process further.
> The above table shows the cluster usage before the balancer run and after
> balancer run using 1 as threshold. After balancer completion, still n1 is
> over utilized and n4 is under utilized. This may be because of n4 contains
> all the blocks which are present in n1. I feel this can be improved further
> by moving data from over utilized nodes to balanced nodes and then balanced
> nodes to under utilized nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
