I realized the main concern with this proposal is how user can interpret this CPU-percentage based quota. Since this quota is exposed to user, we need to explain to user how this quota is going to impact their application performance and convince them that the quota is now too low for their application. We can able to do this with byte-rate based quota. But I am not sure how we can do this with CPU-percentage based quota. For example, how is user going to understand whether 1% CPU is OK?
On Fri, Feb 17, 2017 at 10:11 AM, Onur Karaman <onurkaraman.apa...@gmail.com > wrote: > Overall a big fan of the KIP. > > I'd have to agree with Dong. I'm not sure about the decision of using the > percentage over the window as opposed to request rate. It's pretty hard to > reason about. I just spoke to one of our SRE's and he agrees. > > Also I may have missed it, but I couldn't find information in the KIP on > where this window would be configured. > > On Fri, Feb 17, 2017 at 9:45 AM, Dong Lin <lindon...@gmail.com> wrote: > > > To correct the typo above: It seems to me that determination of request > > rate is not any more difficult than determination of *byte* rate as both > > metrics are commonly used to measure performance and provide guarantee to > > user. > > > > On Fri, Feb 17, 2017 at 9:40 AM, Dong Lin <lindon...@gmail.com> wrote: > > > > > Hey Rajini, > > > > > > Thanks for the KIP. I have some questions: > > > > > > - I am wondering why throttling based on request rate is listed as a > > > rejected alternative. Can you provide more specific reason why it is > > > difficult for administrators to decide request rates to allocate? It > > seems > > > to me that determination of request rate is not any more difficult than > > > determination of request rate as both metrics are commonly used to > > measure > > > performance and provide guarantee to user. On the other hand, the > > > percentage of processing time provides a vague guarantee to user. For > > > example, what performance can user expect if you provide 1% processing > > time > > > quota to this user? How is administrator going to decide this quota? > > Should > > > Kafka administrator continues to reduce this percentage quota as number > > of > > > users grow? > > > > > > - The KIP suggests that LeaderAndIsrRequest and MetadataRequest will > also > > > be throttled by this quota. What is the motivation for throttling these > > > requests? It is also inconsistent with rate-based quota which is only > > > applied to ProduceRequest and FetchRequest. IMO it will be simpler to > > only > > > throttle ProduceRequest and FetchRequest. > > > > > > - Do you think we should also throttle the inter-broker traffic using > > this > > > quota as well similar to KIP-73? > > > > > > Thanks, > > > Dong > > > > > > > > > > > > On Fri, Feb 17, 2017 at 9:05 AM, Rajini Sivaram < > rajinisiva...@gmail.com > > > > > > wrote: > > > > > >> Hi all, > > >> > > >> I have just created KIP-124 to introduce request rate quotas to Kafka: > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-124+-+ > > >> Request+rate+quotas > > >> > > >> The proposal is for a simple percentage request handling time quota > that > > >> can be allocated to *<client-id>*, *<user>* or *<user, client-id>*. > > There > > >> are a few other suggestions also under "Rejected alternatives". > Feedback > > >> and suggestions are welcome. > > >> > > >> Thank you... > > >> > > >> Regards, > > >> > > >> Rajini > > >> > > > > > > > > >
