[
https://issues.apache.org/jira/browse/HDFS-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095615#comment-16095615
]
Konstantin Shvachko commented on HDFS-11742:
--------------------------------------------
Hey [~kihwal], I tested the 2.7.4 Balancer on a small cluster (120 nodes) with
and without your patch. I could not reproduce the degradation of performance.
The ReplaceBlockOpNumOps were the same in both cases, lower than on the large
cluster, but not anything close to the lows as on your graph. Sorry to say
this, but I tried. Probably a different type of imbalance or configuration
parameters. So I don't see this as a blocker.
But I also reviewed your patch and found the change reasonable, as I mentioned
before. So we can commit it, especially as you feel strong about it. My formal
+1 for the patch.
It is better to commit it promptly. I am waiting on the last blocker HDFS-11896
before starting the RC process.
> Improve balancer usability after HDFS-8818
> ------------------------------------------
>
> Key: HDFS-11742
> URL: https://issues.apache.org/jira/browse/HDFS-11742
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Labels: release-blocker
> Attachments: balancer2.8.png, balancer_fix.png,
> HDFS-11742.branch-2.8.patch, HDFS-11742.branch-2.patch,
> HDFS-11742.trunk.patch, HDFS-11742.v2.trunk.patch, replaceBlockNumOps-8w.jpg
>
>
> We ran 2.8 balancer with HDFS-8818 on a 280-node and a 2,400-node cluster. In
> both cases, it would hang forever after two iterations. The two iterations
> were also moving things at a significantly lower rate. The hang itself is
> fixed by HDFS-11377, but the design limitation remains, so the balancer
> throughput ends up actually lower.
> Instead of reverting HDFS-8188 as originally suggested, I am making a small
> change to make it less error prone and more usable.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
