Hello Tao, As I understand, there is a fine balance between rate-limiting, backpressure and not keeping clients waiting. Different use cases may need different approach to rate-limiting and thus, making rate limiter customizable is my first step towards making pulsar more customizable as per need.
Regards On Fri, Nov 3, 2023 at 5:42 PM 太上玄元道君 <ariestp...@gmail.com> wrote: > Hi Girish, > > There is also a discussion thread[1] about rate-limiting. > > I think there is some conflicts between some kind of rate-limiter and > backpressure > > Take the fail-fast strategy as an example: > Brokers have to reply to clients after receiving and decode the message, > but the broker also has the back-pressure mechanism. Broker cannot read > messages because the channel is `disableAutoRead`. > > So the rate-limiters have to adapt to back-pressure. > > Thanks, > Tao Jiuming > > 2023年10月19日 20:51，Girish Sharma <scrapmachi...@gmail.com> 写道： > > Hi, > Currently, there are only 2 kinds of publish rate limiters - polling based > and precise. Users have an option to use either one of them in the topic > publish rate limiter, but the resource group rate limiter only uses polling > one. > > There are challenges with both the rate limiters and the fact that we can't > use precise rate limiter in the resource group level. > > Thus, in order to support custom rate limiters, I've created the PIP-310 > > This is the discussion thread. Please go through the PIP and provide your > inputs. > > Link - https://github.com/apache/pulsar/pull/21399 > > Regards > -- > Girish Sharma > -- Girish Sharma
