As I mentioned in the implementation of PIP, we will plug-in the partition assignment strategy.
However, in the same cluster, it is impossible for some Brokers to use consistent hashing and some Brokers to use round robin. On 2023/04/11 07:37:19 Xiangying Meng wrote: > Hi Linlin, > > This is an incompatible modification, so the entire cluster needs to be > upgraded, not just a part of the nodes > > Appreciate your contribution to the new feature in PIP-255. > I have a question regarding the load-balancing aspect of this feature. > > You mentioned that this is an incompatible modification, > and the entire cluster needs to be upgraded, not just a part of the nodes. > I was wondering why we can only have one load-balancing strategy. > Would it be possible to abstract the logic here and make it an optional > choice? > This way, we could have multiple load-balancing strategies, > such as hash-based, round-robin, etc., available for users to choose from. > > I'd love to hear your thoughts on this. > > Best regards, > Xiangying > > On Mon, Apr 10, 2023 at 8:23 PM PengHui Li <peng...@apache.org> wrote: > > > Hi Lin, > > > > > The load managed by each Bundle is not even. Even if the number of > > partitions managed > > by each bundle is the same, there is no guarantee that the sum of the > > loads of these partitions > > will be the same. > > > > Do we expect that the bundles should have the same loads? The bundle is the > > base unit of the > > load balancer, we can set the high watermark of the bundle, e.g., the > > maximum topics and throughput. > > But the bundle can have different real loads, and if one bundle runs out of > > the high watermark, the bundle > > will be split. Users can tune the high watermark to distribute the loads > > evenly across brokers. > > > > For example, there are 4 bundles with loads 1, 3, 2, 4, the maximum load of > > a bundle is 5 and 2 brokers. > > We can assign bundle 0 and bundle 3 to broker-0 and bundle 1 and bundle 2 > > to broker-2. > > > > Of course, this is the ideal situation. If bundle 0 has been assigned to > > broker-0 and bundle 1 has been > > assigned to broker-1. Now, bundle 2 will go to broker 1, and bundle 3 will > > go to broker 1. The loads for each > > broker are 3 and 7. Dynamic programming can help to find an optimized > > solution with more bundle unloads. > > > > So, should we design the bundle to have even loads? It is difficult to > > achieve in reality. And the proposal > > said, "Let each bundle carry the same load as possible". Is it the correct > > direction for the load balancer? > > > > > Doesn't shed loads very well. The existing default policy > > ThresholdShedder has a relatively high usage > > threshold, and various traffic thresholds need to be set. Many clusters > > with high TPS and small message > > bodies may have high CPU but low traffic; And for many small-scale > > clusters, the threshold needs to be > > modified according to the actual business. > > > > Can it be resolved by introducing the entry write/read rate to the bundle > > stats? > > > > > The removed Bundle cannot be well distributed to other Brokers. The load > > information of each Broker > > will be reported at regular intervals, so the judgment of the Leader > > Broker when allocating Bundles cannot > > be guaranteed to be completely correct. Secondly, if there are a large > > number of Bundles to be redistributed, > > the Leader may make the low-load Broker a new high-load node when the > > load information is not up-to-date. > > > > Can we try to force-sync the load data of the brokers before performing the > > distribution of a large number of > > bundles? > > > > For the Goal section in the proposal. It looks like it doesn't map to the > > issues mentioned in the Motivation section. > > IMO, the proposal should clearly describe the Goal, like which problem will > > be resolved with this proposal. > > Both of the above 3 issues or part of them. And what is the high-level > > solution to resolve the issue, > > and what are the pros and cons compared with the existing solution without > > diving into the implementation section. > > > > Another consideration is the default max bundles of a namespace is 128. I > > don't think the common cases that need > > to set 128 partitions for a topic. If the partitions < the bundle's count, > > will the new solution basically be equivalent to > > the current way? > > > > If this is not a general solution for common scenarios. I support making > > the topic-bundle assigner pluggable without > > introducing the implementation to the Pulsar repo. Users can implement > > their own assigner based on the business > > requirement. Pulsar's general solution may not be good for all scenarios, > > but it is better for scalability (bundle split) > > and enough for most common scenarios. We can keep improving the general > > solution for the general requirement > > for the most common scenarios. > > > > Regards, > > Penghui > > > > > > On Wed, Mar 22, 2023 at 9:52 AM Lin Lin <lin...@apache.org> wrote: > > > > > > > > > This appears to be the "round-robin topic-to-bundle mapping" option in > > > > the `fundBundle` function. Is this the only place that needs an update? > > > Can > > > > you list what change is required? > > > > > > In this PIP, we only discuss topic-to-bundle mapping > > > Change is required: > > > 1) > > > When lookup, partitions is assigned to bundle: > > > Lookup -> NamespaceService#getBrokerServiceUrlAsync -> > > > NamespaceService#getBundleAsync -> > > > NamespaceBundles#findBundle > > > Consistent hashing is now used to assign partitions to bundle in > > > NamespaceBundles#findBundle. > > > We should add a configuration item partitionAssignerClassName, so that > > > different partition assignment algorithms can be dynamically configured. > > > The existing algorithm will be used as the default > > > （partitionAssignerClassName=ConsistentHashingPartitionAssigner） > > > 2) > > > Implement a new partition assignment class RoundRobinPartitionAssigner. > > > New partition assignments will be implemented in this class > > > > > > > > > > How do we enable this "round-robin topic-to-bundle mapping option" (by > > > > namespace policy and broker.conf)? > > > > > > In broker.conf, a new option called `partitionAssignerClassName` > > > > > > > Can we apply this option to existing namespaces? (what's the admin > > > > operation to enable this option)? > > > > > > The cluster must ensure that all nodes use the same algorithm. > > > Broker-level configuration can be made effective by restarting or admin > > API > > > BrokersBase#updateDynamicConfiguration > > > > > > > I assume the "round-robin topic-to-bundle mapping option" works with a > > > > single partitioned topic, because other topics might show different > > load > > > > per partition. Is this intention? (so users need to ensure not to put > > > other > > > > topics in the namespace, if this option is configured) > > > > > > For single-partition topics, since the starting bundle is determined > > > using a consistent hash. > > > Therefore, single-partition topics will spread out to different bundle > > as > > > much as possible. > > > For high load single-partition topics, current algorithms cannot solve > > > this problem. > > > This PIP cannot solve this problem as well. > > > If it just a low load single-partition topic , the impact on the entire > > > bundle is very small. > > > However, in real scenarios, high-load businesses will share the load > > > through multiple partitions. > > > > > > > Some brokers might have more bundles than other brokers. Do we have > > > > different logic for bundle balancing across brokers? or do we rely on > > the > > > > existing assign/unload/split logic to balance bundles among brokers? > > > > > > In this PIP, we do not involve the mapping between bundles and brokers, > > > the existing algorithm works well with this PIP. > > > However, we will also contribute our mapping algorithm in the subsequent > > > PIP. > > > For example: bundles under same namespace can be assigned to broker in a > > > round-robin manner. > > > > > > > > > > > >
