This proposal is easier to understand than before. Overall LGTM. But I think these `onBundleXXX` methods could be default so that we can implement it with a simple lambda.
Thanks, Yunze On Wed, Apr 19, 2023 at 10:22 AM Lin Lin <lin...@apache.org> wrote: > > We make this configuration item to be a dynamic configuration. > We change change it on broker level. > If we can change it on namespace level, even load of bundle in some namespace > is balanced, it is still difficult to make broker balance > > On 2023/04/16 16:07:45 lifepuzzlefun wrote: > > I think this feature is very helpful on heavy traffic topic which have > > continuous stable load on each partition. > > > > > > Is there a way we can set some kind of namespace policy to set the plugin > > PartitionAssigner. Hope this can be set on namespace level, > > if this can be achieved, it is more adoptable to try this feature in > > production environment. : - ) > > > > At 2023-04-12 11:24:11, "Lin Lin" <lin...@apache.org> wrote: > > >As I mentioned in the implementation of PIP, we will plug-in the partition > > >assignment strategy. > > > > > >However, in the same cluster, it is impossible for some Brokers to use > > >consistent hashing and some Brokers to use round robin. > > > > > >On 2023/04/11 07:37:19 Xiangying Meng wrote: > > >> Hi Linlin, > > >> > This is an incompatible modification, so the entire cluster needs to be > > >> upgraded, not just a part of the nodes > > >> > > >> Appreciate your contribution to the new feature in PIP-255. > > >> I have a question regarding the load-balancing aspect of this feature. > > >> > > >> You mentioned that this is an incompatible modification, > > >> and the entire cluster needs to be upgraded, not just a part of the > > >> nodes. > > >> I was wondering why we can only have one load-balancing strategy. > > >> Would it be possible to abstract the logic here and make it an optional > > >> choice? > > >> This way, we could have multiple load-balancing strategies, > > >> such as hash-based, round-robin, etc., available for users to choose > > >> from. > > >> > > >> I'd love to hear your thoughts on this. > > >> > > >> Best regards, > > >> Xiangying > > >> > > >> On Mon, Apr 10, 2023 at 8:23 PM PengHui Li <peng...@apache.org> wrote: > > >> > > >> > Hi Lin, > > >> > > > >> > > The load managed by each Bundle is not even. Even if the number of > > >> > partitions managed > > >> > by each bundle is the same, there is no guarantee that the sum of > > >> > the > > >> > loads of these partitions > > >> > will be the same. > > >> > > > >> > Do we expect that the bundles should have the same loads? The bundle > > >> > is the > > >> > base unit of the > > >> > load balancer, we can set the high watermark of the bundle, e.g., the > > >> > maximum topics and throughput. > > >> > But the bundle can have different real loads, and if one bundle runs > > >> > out of > > >> > the high watermark, the bundle > > >> > will be split. Users can tune the high watermark to distribute the > > >> > loads > > >> > evenly across brokers. > > >> > > > >> > For example, there are 4 bundles with loads 1, 3, 2, 4, the maximum > > >> > load of > > >> > a bundle is 5 and 2 brokers. > > >> > We can assign bundle 0 and bundle 3 to broker-0 and bundle 1 and > > >> > bundle 2 > > >> > to broker-2. > > >> > > > >> > Of course, this is the ideal situation. If bundle 0 has been assigned > > >> > to > > >> > broker-0 and bundle 1 has been > > >> > assigned to broker-1. Now, bundle 2 will go to broker 1, and bundle 3 > > >> > will > > >> > go to broker 1. The loads for each > > >> > broker are 3 and 7. Dynamic programming can help to find an optimized > > >> > solution with more bundle unloads. > > >> > > > >> > So, should we design the bundle to have even loads? It is difficult to > > >> > achieve in reality. And the proposal > > >> > said, "Let each bundle carry the same load as possible". Is it the > > >> > correct > > >> > direction for the load balancer? > > >> > > > >> > > Doesn't shed loads very well. The existing default policy > > >> > ThresholdShedder has a relatively high usage > > >> > threshold, and various traffic thresholds need to be set. Many > > >> > clusters > > >> > with high TPS and small message > > >> > bodies may have high CPU but low traffic; And for many small-scale > > >> > clusters, the threshold needs to be > > >> > modified according to the actual business. > > >> > > > >> > Can it be resolved by introducing the entry write/read rate to the > > >> > bundle > > >> > stats? > > >> > > > >> > > The removed Bundle cannot be well distributed to other Brokers. The > > >> > > load > > >> > information of each Broker > > >> > will be reported at regular intervals, so the judgment of the Leader > > >> > Broker when allocating Bundles cannot > > >> > be guaranteed to be completely correct. Secondly, if there are a > > >> > large > > >> > number of Bundles to be redistributed, > > >> > the Leader may make the low-load Broker a new high-load node when > > >> > the > > >> > load information is not up-to-date. > > >> > > > >> > Can we try to force-sync the load data of the brokers before > > >> > performing the > > >> > distribution of a large number of > > >> > bundles? > > >> > > > >> > For the Goal section in the proposal. It looks like it doesn't map to > > >> > the > > >> > issues mentioned in the Motivation section. > > >> > IMO, the proposal should clearly describe the Goal, like which problem > > >> > will > > >> > be resolved with this proposal. > > >> > Both of the above 3 issues or part of them. And what is the high-level > > >> > solution to resolve the issue, > > >> > and what are the pros and cons compared with the existing solution > > >> > without > > >> > diving into the implementation section. > > >> > > > >> > Another consideration is the default max bundles of a namespace is > > >> > 128. I > > >> > don't think the common cases that need > > >> > to set 128 partitions for a topic. If the partitions < the bundle's > > >> > count, > > >> > will the new solution basically be equivalent to > > >> > the current way? > > >> > > > >> > If this is not a general solution for common scenarios. I support > > >> > making > > >> > the topic-bundle assigner pluggable without > > >> > introducing the implementation to the Pulsar repo. Users can implement > > >> > their own assigner based on the business > > >> > requirement. Pulsar's general solution may not be good for all > > >> > scenarios, > > >> > but it is better for scalability (bundle split) > > >> > and enough for most common scenarios. We can keep improving the general > > >> > solution for the general requirement > > >> > for the most common scenarios. > > >> > > > >> > Regards, > > >> > Penghui > > >> > > > >> > > > >> > On Wed, Mar 22, 2023 at 9:52 AM Lin Lin <lin...@apache.org> wrote: > > >> > > > >> > > > > >> > > > This appears to be the "round-robin topic-to-bundle mapping" > > >> > > > option in > > >> > > > the `fundBundle` function. Is this the only place that needs an > > >> > > > update? > > >> > > Can > > >> > > > you list what change is required? > > >> > > > > >> > > In this PIP, we only discuss topic-to-bundle mapping > > >> > > Change is required: > > >> > > 1) > > >> > > When lookup, partitions is assigned to bundle: > > >> > > Lookup -> NamespaceService#getBrokerServiceUrlAsync -> > > >> > > NamespaceService#getBundleAsync -> > > >> > > NamespaceBundles#findBundle > > >> > > Consistent hashing is now used to assign partitions to bundle in > > >> > > NamespaceBundles#findBundle. > > >> > > We should add a configuration item partitionAssignerClassName, so > > >> > > that > > >> > > different partition assignment algorithms can be dynamically > > >> > > configured. > > >> > > The existing algorithm will be used as the default > > >> > > （partitionAssignerClassName=ConsistentHashingPartitionAssigner） > > >> > > 2) > > >> > > Implement a new partition assignment class > > >> > > RoundRobinPartitionAssigner. > > >> > > New partition assignments will be implemented in this class > > >> > > > > >> > > > > >> > > > How do we enable this "round-robin topic-to-bundle mapping option" > > >> > > > (by > > >> > > > namespace policy and broker.conf)? > > >> > > > > >> > > In broker.conf, a new option called `partitionAssignerClassName` > > >> > > > > >> > > > Can we apply this option to existing namespaces? (what's the admin > > >> > > > operation to enable this option)? > > >> > > > > >> > > The cluster must ensure that all nodes use the same algorithm. > > >> > > Broker-level configuration can be made effective by restarting or > > >> > > admin > > >> > API > > >> > > BrokersBase#updateDynamicConfiguration > > >> > > > > >> > > > I assume the "round-robin topic-to-bundle mapping option" works > > >> > > > with a > > >> > > > single partitioned topic, because other topics might show different > > >> > load > > >> > > > per partition. Is this intention? (so users need to ensure not to > > >> > > > put > > >> > > other > > >> > > > topics in the namespace, if this option is configured) > > >> > > > > >> > > For single-partition topics, since the starting bundle is determined > > >> > > using a consistent hash. > > >> > > Therefore, single-partition topics will spread out to different > > >> > > bundle > > >> > as > > >> > > much as possible. > > >> > > For high load single-partition topics, current algorithms cannot > > >> > > solve > > >> > > this problem. > > >> > > This PIP cannot solve this problem as well. > > >> > > If it just a low load single-partition topic , the impact on the > > >> > > entire > > >> > > bundle is very small. > > >> > > However, in real scenarios, high-load businesses will share the load > > >> > > through multiple partitions. > > >> > > > > >> > > > Some brokers might have more bundles than other brokers. Do we have > > >> > > > different logic for bundle balancing across brokers? or do we rely > > >> > > > on > > >> > the > > >> > > > existing assign/unload/split logic to balance bundles among > > >> > > > brokers? > > >> > > > > >> > > In this PIP, we do not involve the mapping between bundles and > > >> > > brokers, > > >> > > the existing algorithm works well with this PIP. > > >> > > However, we will also contribute our mapping algorithm in the > > >> > > subsequent > > >> > > PIP. > > >> > > For example: bundles under same namespace can be assigned to broker > > >> > > in a > > >> > > round-robin manner. > > >> > > > > >> > > > > >> > > > > >> > > > >> > >
