Jiajun Wang created HELIX-672:
---------------------------------
Summary: Improve partition distribution strategy for small clusters
Key: HELIX-672
URL: https://issues.apache.org/jira/browse/HELIX-672
Project: Apache Helix
Issue Type: Task
Reporter: Jiajun Wang
Assignee: Jiajun Wang
Current, CRUSH (also multi-round CRUSH) strategy provides even partition
distribution based on statistically randomization. All CRUSH based strategy
works well when the cluster is reasonably large.
However, in some of our small production clusters, we find the distribution is
not good as expected. The difference between heavy load node and idle node
could be 30% in our case. Given we assign resource according to the max load,
we are wasting 30% of resource on the idle nodes. And it could be worse in
other clusters.
We need to find a new algorithm or improve CRUSH for better evenness for small
clusters.
Note that, meanwhile, we should keep the good features such as minimum
movements, deterministic calculating result.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
