The proposal in KIP-55 is to have hierarchical quotas with two levels of
hierarchy - quotas assigned to users with sub-quotas assigned to clients
(client-ids) of users. Typically, clients within a JVM run as a single
user, but within a JVM, clients could use different client-ids for the
purpose of metrics and quotas. The goal is to support the same quota use
cases for authenticated users as with client-id, but in a secure way. The
difference however is that client1 of UserA is treated a different client
from client1 of UserB w.r.t quota when the new scheme is used. I am hoping
that is a reasonable deviation.
This discussion suggests that perhaps another level in the hierarchy that
groups clients together for quotas would be useful in certain scenarios. An
alternative to an unauthenticated quota-id field in client requests would
be to define quotas in terms of three levels of hierarchy:
- UserA
- ClientGroup1
- Client1
- Client2
- ClientGroup2
- Client3
- Client4
Sub-quotas can be defined at any of the levels of the hierarchy - for
example, Client1 and Client2 could have their own sub-quotas or share the
quota of ClientGroup1. For unauthenticated users, this will revert to two
levels of hierarchy (client-group, client-id). Clients without a group will
be treated as a client-group, giving the current quota configuration based
on client-ids. Rather than introduce another id in requests that can be set
to anything by clients, this would enable client-ids to be grouped together
at the broker-side. And conceptually, it would be a similar grouping for
authenticated and unauthenticated clients.
Thoughts?
On Fri, Apr 29, 2016 at 11:54 PM, Joel Koshy <jjkosh...@gmail.com> wrote:
> Currently, the only available mechanism to disambiguate between
> producer/consumer clients in the same JVM is the client-id. However, that
> does not play very well with the current definition of client-id and its
> use as the entity for quota enforcement. i.e., an application can thwart
> quota enforcement by using multiple client-ids (either insidiously to go
> beyond its allocated quota or with good intention to avoid the metric
> collision issue)
>
> Name mangling works but I think less useful than an explicit metric-id
> (perhaps that could be an optional configuration) in that we would need to
> somehow correlate the mangled name with the actual code (say, when looking
> at a threaddump or looking at request logs).
>
> btw, the proposal in KIP-55 would work for authenticated clients - i.e.,
> the application can instantiate multiple clients with different client-ids
> but same quota id (authenticated user) and avoids a wire-protocol change,
> but we ideally should have something that also works for non-authenticated
> clients. Between an explicit quota-id field and an instance disambiguation
> scheme (i.e., metric id) I would vote for the latter.
>
> On Fri, Apr 29, 2016 at 1:06 AM, Onur Karaman <
> onurkaraman.apa...@gmail.com>
> > wrote:
>
> > alternative is to make quotas depend on a quota id instead of client-id
> > (as
> > > brought up in KIP-55), but this means we no longer can quota older
> > clients
> > > out of the box.
> >
>
> If we do have a new quota-id then we can still quota older clients - i.e.,
> if the inbound request has a newer API version that has a quota-id then use
> that - otherwise use the client-id.
>
> Joel
>
--
Regards,
Rajini
