cmccabe commented on PR #13116: URL: https://github.com/apache/kafka/pull/13116#issuecomment-1400816232
> Also note that I didn't implement the full logic -- QuorumController has additional checks that could in fact cause a specific topic request to be rejected (e.g. explicit assignments not contiguous partition numbers starting with 0). This could lead to a calculation of partition counts for throttling purposes that exceeds the actual partition counts requested -- which could cause throttling in cases where technically it should not. I think arguably if you are making requests that fail, your quota should be dinged for those requests. Surely we don't want to let the guy spinning in a tight loop making tons of bad requests go un-throttled? I would even maybe include requests that fail authorization in the quota, although I know the ZK implementation might not. Authorization is expensive; you should get charged for wasting the authorizer's time with bad rqeuests. Maybe we should charge for validateOnly requests as well. They are almost as expensive as non-validateOnly requests... the only difference is no records written. > The benefit of the Scala approach is that we avoid doing work in the controller itself when throttling does occur. We don't generate all of the Metadata Log records, for example. Note that createPartitions and delateTopics will have to get access to the partition counts, which currently I think is not available. One big issue is that ControllerMutationQuota takes some locks, and I am worried that the controller thread could get blocked behind those locks. This is similar to the Authorizer where for better or worse, we went with a blocking implementation that would be too slow to call from the quorum controller thread (for example, the Authorizer implementation might send out a blocking request to ZK or other external system, etc.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
