One additional question: In your use case, do you have multiple producers concurrently producing to the same topic from different clients?
The use case is challenging in the current implementation when using topic producing rate limiting. The problem is that the different producers will be able to send messages at very different rates since there isn't a solution to ensure fairness across multiple producers in the topic producer rate limiting solution. This is something that should be addressed when improving rate limiting. -Lari la 4. marrask. 2023 klo 17.24 Lari Hotari <lhot...@apache.org> kirjoitti: > Replies inline > > On Fri, 3 Nov 2023 at 20:48, Girish Sharma <scrapmachi...@gmail.com> > wrote: > >> Could you please elaborate more on these details? Here are some questions: >> > 1. What do you mean that it is too strict? >> > - Should the rate limiting allow bursting over the limit for some >> time? >> > >> >> That's one of the major use cases, yes. >> > > One possibility would be to improve the existing rate limiter to allow > bursting. > I think that Pulsar's out-of-the-box rate limiter should cover 99% of the > use cases instead of having one implementing their own rate limiter > algorithm. > The problems you are describing seem to be common to many Pulsar use > cases, and therefore, I think they should be handled directly in Pulsar. > > Optimally, there would be a single solution that abstracts the rate > limiting in a way where it does the right thing based on the declarative > configuration. > I would prefer that over having a pluggable solution for rate limiter > implementations. > > What would help is getting deeper in the design of the rate limiter > itself, without limiting ourselves to the existing rate limiter > implementation in Pulsar. > > In textbooks, there are algorithms such as "leaky bucket" [1] and "token > bucket" [2]. Both algorithms have several variations and in some ways they > are very similar algorithms but looking from the different point of view. > It would possibly be easier to conceptualize and understand a rate limiting > algorithm if common algorithm names and implementation choices mentioned in > textbooks would be referenced in the implementation. > It seems that a "token bucket" type of algorithm can be used to implement > rate limiting with bursting. In the token bucket algorithm, the size of the > token bucket defines how large bursts will be allowed. The design could > also be something where 2 rate limiters with different type of algorithms > and/or configuration parameters are combined to achieve a desired behavior. > For example, to achieve a rate limiter with bursting and a fixed maximum > rate. > By default, the token bucket algorithm doesn't enforce a maximum rate for > bursts, but that could be achieved by chaining 2 rate limiters if that is > really needed. > > The current Pulsar rate limiter implementation could be implemented in a > cleaner way, which would also be more efficient. Instead of having a > scheduler call a method once per second, I think that the rate limiter > could be implemented in a reactive way where the algorithm is implemented > without a scheduler. > I wonder if there are others that would be interested in getting down into > such implementation details? > > 1 - https://en.wikipedia.org/wiki/Leaky_bucket > 2 - https://en.wikipedia.org/wiki/Token_bucket > > > > 2. What type of data loss are you experiencing? >> > >> >> Messages produced by the producers which eventually get timed out due to >> rate limiting. >> > > Are you able to slow down producing on the client side? If that is > possible, there could be ways to improve ways to do client side back > pressure with Pulsar Client. Currently, the client doesn't expose this > information until the sending blocks or fails with an exception > (ProducerQueueIsFullError). Optimally, the client should slow down the rate > of producing to the rate that it can actually send to the broker. > Just curious if you have considered turning off producing timeouts on the > client side completely or making them longer? Would that address the data > loss problem? > Or is your event/message source "hot" so that you cannot stop or slow it > down, and it will just keep on flowing with a certain rate? > > > I think the core implementation of how the broker fails fast at the time >> of rate limiting (whether it is by pausing netty channel or a new permits >> based model) does not change the actual issue I am targeting. Multiplexing >> has some impact on it - but yet again only limited, and can easily be >> fixed >> by the client by increasing the connections per broker. Even after >> assuming >> both these things are somehow "fixed", the fact remains that an absolutely >> strict rate limiter will lead to the above mentioned data loss for burst >> going above the limit and that a poller based rate limiter doesn't really >> rate limit anything as it allows all produce in the first interval of the >> next second. >> > > Yes, it makes sense to have bursting configuration parameters in the rate > limiter. > As mentioned above, I think we could be improving the existing rate > limiter in Pulsar to cover 99% of the use case by making it stable and by > including the bursting configuration options. > Is there additional functionality you feel the rate limiter needs beyond > bursting support? > > One way to workaround the multiplexing problem would be to add a client > side option for producers and consumers, where you could specify that the > client picks a separate TCP/IP connection that is not shared and isn't from > the connection pool. > Preventing connection multiplexing seems to be the only way to make the > current rate limiting deterministic and stable without adding the explicit > flow control to the Pulsar binary protocol for producers. > > Are there other community members with input on the design and > implementation of an improved rate limiter? > I’m eager to continue this conversation and work together towards a robust > solution. > > -Lari >
