[
https://issues.apache.org/jira/browse/HBASE-21014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572945#comment-16572945
]
Toshihiro Suzuki edited comment on HBASE-21014 at 8/8/18 9:43 AM:
------------------------------------------------------------------
Hi [~harisekhon],
I think we can't avoid to lose data localities for HBase when you run the HDFS
balancer. This is because HDFS doesn't know the region locations and it doesn't
take the locations into account for block balancing. This is the same even when
you use FavoredNodeLoadBalancer. If you need to run HDFS balancer, you can run
major compaction after that to recover the data localities.
Thanks.
was (Author: brfrn169):
Hi [~harisekhon],
I think we can't avoid to lose data localities for HBase when you run the HDFS
balancer. This is because HDFS doesn't know the region locations and it doesn't
take the locations into account for block balancing. This is the same even when
you use FavoredNodeLoadBalancer. If you need to run HDFS balancer, we can run
major compaction after that to recover the data localities.
Thanks.
> Improve Stochastic Balancer to write HDFS favoured node hints for region
> primary blocks to avoid destroying data locality if needing to use HDFS
> Balancer
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-21014
> URL: https://issues.apache.org/jira/browse/HBASE-21014
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
> Affects Versions: 1.1.2
> Reporter: Hari Sekhon
> Priority: Major
>
> Improve Stochastic Balancer to include the HDFS region location hints to
> avoid HDFS Balancer destroying data locality.
> Right now according to a mix of docs, jiras and mailing list info it appears
> that one must change
> {code:java}
> hbase.master.loadbalancer.class{code}
> to the org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer as it looks
> like this functionality is only within FavoredNodeBalancer and not the
> standard Stochastic Balancer.
> [http://hbase.apache.org/book.html#_hbase_and_hdfs]
> This is not ideal because we'd still like to use all the heuristics and work
> that has gone in the Stochastic Balancer which I believe right now is the
> best and most mature HBase balancer.
> See also the linked Jiras and this discussion:
> [http://apache-hbase.679495.n3.nabble.com/HDFS-Balancer-td4086607.html]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
