> This appears to be the "round-robin topic-to-bundle mapping" option in > the `fundBundle` function. Is this the only place that needs an update? Can > you list what change is required?
In this PIP, we only discuss topic-to-bundle mapping Change is required: 1) When lookup, partitions is assigned to bundle: Lookup -> NamespaceService#getBrokerServiceUrlAsync -> NamespaceService#getBundleAsync -> NamespaceBundles#findBundle Consistent hashing is now used to assign partitions to bundle in NamespaceBundles#findBundle. We should add a configuration item partitionAssignerClassName, so that different partition assignment algorithms can be dynamically configured. The existing algorithm will be used as the default （partitionAssignerClassName=ConsistentHashingPartitionAssigner） 2) Implement a new partition assignment class RoundRobinPartitionAssigner. New partition assignments will be implemented in this class > How do we enable this "round-robin topic-to-bundle mapping option" (by > namespace policy and broker.conf)? In broker.conf, a new option called `partitionAssignerClassName` > Can we apply this option to existing namespaces? (what's the admin > operation to enable this option)? The cluster must ensure that all nodes use the same algorithm. Broker-level configuration can be made effective by restarting or admin API BrokersBase#updateDynamicConfiguration > I assume the "round-robin topic-to-bundle mapping option" works with a > single partitioned topic, because other topics might show different load > per partition. Is this intention? (so users need to ensure not to put other > topics in the namespace, if this option is configured) For single-partition topics, since the starting bundle is determined using a consistent hash. Therefore, single-partition topics will spread out to different bundle as much as possible. For high load single-partition topics, current algorithms cannot solve this problem. This PIP cannot solve this problem as well. If it just a low load single-partition topic , the impact on the entire bundle is very small. However, in real scenarios, high-load businesses will share the load through multiple partitions. > Some brokers might have more bundles than other brokers. Do we have > different logic for bundle balancing across brokers? or do we rely on the > existing assign/unload/split logic to balance bundles among brokers? In this PIP, we do not involve the mapping between bundles and brokers, the existing algorithm works well with this PIP. However, we will also contribute our mapping algorithm in the subsequent PIP. For example: bundles under same namespace can be assigned to broker in a round-robin manner.
