Hi, David, Anna, Thanks for the response. Sorry for the late reply.
10. Regarding exposing rate or usage as quota. Your argument is that usage is not very accurate anyway and is harder to implement. So, let's just be a bit loose and expose rate. I am sort of neutral on that. (1) It seems to me that overall usage will be relatively more accurate than rate. All the issues that Anna brought up seem to exist for rate too. Rate has the additional problem that the cost of each request may not be uniform. (2) In terms of implementation, a usage based approach requires tracking the user info through the life cycle of an operation. However, as you mentioned, things like topic creation can generate additional LeaderAndIsr/UpdateMetadata requests. Longer term, we probably want to associate those cost to the user who initiated the operation. If we do that, we sort of need to track the user for the full life cycle of the processing of an operation anyway. (3) If you prefer rate strongly, I don't have strong objections. However, I do feel that the new quota name should be able to cover all controller related cost longer term. This KIP currently only covers topic creation/deletion. It would not be ideal if in the future, we have to add yet another type of quota for some other controller related operations. The quota name in the KIP has partition mutation. In the future, if we allow things like topic renaming, it may not be related to partition mutation directly and it would be trickier to fit those operations in the current quota. So, maybe sth more general like controller_operations_quota will be more future proof. 11. Regarding the difference between the token bucket algorithm and our current quota mechanism. That's a good observation. It seems that we can make a slight change to our current quota mechanism to achieve a similar thing. As you said, the issue was that we allocate all 7 * 80 requests in the last 1 sec window in our current mechanism. This is a bit unintuitive since each sec only has a quota capacity of 5. An alternative way is to allocate the 7 * 80 requests to all previous windows, each up to the remaining quota capacity left. So, you will fill the current 1 sec window and all previous ones, each with 5. Then, it seems this will give the same behavior as token bucket? The reason that I keep asking this is that from an operational perspective, it's simpler if all types of quotas work in the same way. Jun On Tue, May 26, 2020 at 9:52 AM David Jacot <dja...@confluent.io> wrote: > Hi folks, > > I have updated the KIP. As mentioned by Jun, I have made the > quota per principal/clientid similarly to the other quotas. I have > also explained how this will work in conjunction with the auto > topics creation. > > Please, take a look at it. I plan to call a vote for it in the next few > days if there are no comments in this thread. > > Best, > David > > On Wed, May 13, 2020 at 10:57 AM Tom Bentley <tbent...@redhat.com> wrote: > > > Hi David, > > > > Thanks for the explanation and confirmation that evolving the APIs is not > > off the table in the longer term. > > > > Kind regards, > > > > Tom > > >
