[
https://issues.apache.org/jira/browse/HDFS-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-11742:
------------------------------
Description:
We ran 2.8 balancer with HDFS-8818 on a 280-node and a 2,400-node cluster. In
both cases, it would hang forever after two iterations. The two iterations were
also moving things at a significantly lower rate. The hang itself is fixed by
HDFS-11377, but the design limitation remains, so the balancer throughput ends
up actually lower.
Instead of reverting HDFS-8188 as originally suggested, I am making a small
change to make it less error prone and more usable.
was:
This is to revert the core changes made by HDFS-8818. The reason is explained
in the jira comments. HDFS-8818 put in config and logging changes that are
tied to the core change. I will leave them as is.
We ran 2.8 balancer with HDFS-8818 on a 280-node and a 2,400-node cluster. In
both cases, it would hang forever after two iterations. The two iterations were
also moving things at a significantly lower rate. The hang itself is fixed by
HDFS-11377, but the design limitation remains, so the balancer throughput ends
up actually lower.
> Improve balancer usability after HDFS-8188
> ------------------------------------------
>
> Key: HDFS-11742
> URL: https://issues.apache.org/jira/browse/HDFS-11742
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Attachments: balancer2.8.png, HDFS-11742.branch-2.8.patch,
> HDFS-11742.branch-2.patch, HDFS-11742.trunk.patch
>
>
> We ran 2.8 balancer with HDFS-8818 on a 280-node and a 2,400-node cluster. In
> both cases, it would hang forever after two iterations. The two iterations
> were also moving things at a significantly lower rate. The hang itself is
> fixed by HDFS-11377, but the design limitation remains, so the balancer
> throughput ends up actually lower.
> Instead of reverting HDFS-8188 as originally suggested, I am making a small
> change to make it less error prone and more usable.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
