[
https://issues.apache.org/jira/browse/HELIX-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397911#comment-16397911
]
ASF GitHub Bot commented on HELIX-674:
--------------------------------------
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174326343
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util;
+
+import org.apache.helix.HelixException;
+import org.apache.helix.ZNRecord;
+import org.apache.helix.api.config.RebalanceConfig;
+import
org.apache.helix.api.rebalancer.constraint.AbstractRebalanceHardConstraint;
+import
org.apache.helix.api.rebalancer.constraint.AbstractRebalanceSoftConstraint;
+import org.apache.helix.controller.common.PartitionStateMap;
+import org.apache.helix.controller.common.ResourcesStateMap;
+import
org.apache.helix.controller.rebalancer.strategy.ConstraintRebalanceStrategy;
+import org.apache.helix.controller.stages.ClusterDataCache;
+import org.apache.helix.model.*;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * A rebalance tool that generate an resource partition assignment based
on the input.
+ * Note the assignment won't be automatically applied to the cluster.
Applications are supposed to
+ * apply the change.
+ */
+public class WeightAwareRebalanceUtil {
+ private final ClusterConfig _clusterConfig;
+ private final Map<String, InstanceConfig> _instanceConfigMap = new
HashMap<>();
+ // For the possible customized state models.
+ private final Map<String, StateModelDefinition> _stateModelDefs = new
HashMap<>();
+ private final ClusterDataCache _dataCache;
+
+ public enum RebalanceOption {
+ INIT,
+ REASSIGN
+ }
+
+ public WeightAwareRebalanceUtil(ClusterConfig clusterConfig,
+ List<InstanceConfig> instanceConfigs) {
+ for (InstanceConfig instanceConfig : instanceConfigs) {
+ _instanceConfigMap.put(instanceConfig.getInstanceName(),
instanceConfig);
+ }
+ _clusterConfig = clusterConfig;
+
+ _dataCache = new ClusterDataCache();
+ _dataCache.setInstanceConfigMap(_instanceConfigMap);
+ _dataCache.setClusterConfig(_clusterConfig);
+ List<LiveInstance> liveInstanceList = new ArrayList<>();
+ for (String instance : _instanceConfigMap.keySet()) {
+ LiveInstance liveInstance = new LiveInstance(instance);
+ liveInstanceList.add(liveInstance);
+ }
+ _dataCache.setLiveInstances(liveInstanceList);
+ }
+
+ /**
+ * The method to generate partition assignment mappings.
+ *
+ * @param resourceConfigs Config of all the resources that need to be
rebalanced.
+ * The tool throws Exception if any resource
has no IS or broken/uninitialized IS.
+ * The tool throws Exception if any resource
is in full-auto mode.
+ * @param existingAssignment The existing partition assignment of the
input resources.
+ * @param option INIT or REASSIGN
+ * INIT: Keep existing assignment. Only
generate new partition assignment.
--- End diff --
"Mode" is confusing while we already have "RebalanceMode".
About the option name, how about "incremental" rebalance & "full"
rebalance? I think those 2 names are much better.
I will add the comment about the resourceConfig, this is where users should
specify the partitions.
Also, as a standalone tool, most rebalance options won't work even
configured in the cluster config / instance config. I will add comments to
point it out.
> Constraint Based Resource Rebalancer
> ------------------------------------
>
> Key: HELIX-674
> URL: https://issues.apache.org/jira/browse/HELIX-674
> Project: Apache Helix
> Issue Type: New Feature
> Reporter: Jiajun Wang
> Assignee: Jiajun Wang
> Priority: Major
> Fix For: 0.8.x
>
> Attachments: Constraint-BasedResourceRebalancing-080318-2226-240.pdf
>
>
> Helix rebalancer assigns resources according to different strategies.
> Recently, we optimize the strategy for evenness and minimize movement.
> However, the evenness here only applies to partition numbers. Moreover, we've
> got more requests for customizable rebalancer from our users.
> Take partition weight as an example:
> In reality, partition replicas have different size. We use "partition weight"
> as an abstraction of the partition size. It can be network traffic usage,
> disk usage, or any other combined factors.
> Given each partition may have different weights, Helix should be able to
> assign partition accordingly. So that the distribution would be even
> regarding the weight.
> In this project, we are planning new rebalancer mechanism that generates
> resource partition assignment according to a list of "constraints". Current
> rebalance strategy can be regarded as one kind of constraint. Moving forward,
> Helix users would be able to extend the constraint interface using their own
> logic.
> Some init discussions are in progress and we will have a proposal posted here
> soon.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
