I think this proposal makes a lot of sense (especially now that it is oriented around request rate) and fills the biggest remaining gap in the multi-tenancy story.
I think for intra-cluster communication (StopReplica, etc) we could avoid throttling entirely. You can secure or otherwise lock-down the cluster communication to avoid any unauthorized external party from trying to initiate these requests. As a result we are as likely to cause problems as solve them by throttling these, right? I'm not so sure that we should exempt the consumer requests such as heartbeat. It's true that if we throttle an app's heartbeat requests it may cause it to fall out of its consumer group. However if we don't throttle it it may DDOS the cluster if the heartbeat interval is set incorrectly or if some client in some language has a bug. I think the policy with this kind of throttling is to protect the cluster above any individual app, right? I think in general this should be okay since for most deployments this setting is meant as more of a safety valve---that is rather than set something very close to what you expect to need (say 2 req/sec or whatever) you would have something quite high (like 100 req/sec) with this meant to prevent a client gone crazy. I think when used this way allowing those to be throttled would actually provide meaningful protection. -Jay On Fri, Feb 17, 2017 at 9:05 AM, Rajini Sivaram <rajinisiva...@gmail.com> wrote: > Hi all, > > I have just created KIP-124 to introduce request rate quotas to Kafka: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 124+-+Request+rate+quotas > > The proposal is for a simple percentage request handling time quota that > can be allocated to *<client-id>*, *<user>* or *<user, client-id>*. There > are a few other suggestions also under "Rejected alternatives". Feedback > and suggestions are welcome. > > Thank you... > > Regards, > > Rajini >
