Hey Rajini, 1. That makes sense to me. Is that reflected in the documentation anywhere (I couldn't really find it)? Is there a way to discover that definition? We do way better when we right this stuff down so it has an official definition users and developers can work off of... 2. If client id is a thing that makes sense even when you have users, why would you not want to quota on it?
I am not wild about these "modes" where you boot the cluster in mode X and it behaves in one way and in mode Y and it behaves in another. It is complex then for users who expect to be able to set quotas but then have to be able to get access to the filesystem of the kafka nodes to discover what mode the cluster is in to know which kind of quota is applicable. I guess there are two ways to think about a feature: one is the increment from where we are, and the other is the resulting state after that increment is taken. What I am asking is not "is this a low cost step from where we are?" which everyone can agree that it is, but rather "does this make sense as an end state--i.e. if you were starting fresh with neither users nor client ids nor quotas would you end up with this?". In terms of use cases, I think that we support mixing secure and insecure access on a single cluster so presumably in that case you would want to be able to quota insecure users based on client id and secure users based on user, right? Likewise, as you said, client id is a valid grouping even in the presence of users, so it might be the case that several apps that are all part of the same system might access Kafka under a single user, but you might have different quotas for these different apps. Basically if client id is a valid grouping even in the presence of users (willing to debate this point, btw!) then you would want to quota on it. -Jay On Fri, Jun 10, 2016 at 4:49 AM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > Jay, > > Thank you for the feedback. > > 1. I think it is good to have a single concept of identity, but multiple > ways of grouping clients for different functions. Client-id is a logical > grouping of clients with a meaningful name that is used in client metrics > and logs. User principal is an authenticated user or a grouping of > unauthenticated users chosen by the broker and is used for ACLs. In my > view, in a multi-user system, there is a hierarchy - user owns zero or more > clients. (principal, client-id) defines a safe group, but the shorter > unsafe client-id is sufficient in client metrics and logs. > > > 2. KIP-55 was initially written to support hierarchical quotas (quotas for > user1-clientA, user2-clientA etc), but Jun's view was that the complexity > was not justified since there is no clear requirement for this. The > cut-down version of the KIP is clearly a lot simpler. But I think your > suggestion is to enable non-hierarchical user quotas and client-id quotas > at the same time. Basically treat users and client-ids as distinct entities > like topics and allow quotas to be applied to each of these entities. I > agree that we want to support quotas simultaneously on different entities > like topics and users. I am less convinced of client-id and user being > > distinct entities that require separate quotas at the same time. And > treating client-id and user as distinct entities makes it harder to > implement hierarchical quotas in future. > > > A single user system needs only client-id quotas, and multi-tenant system > cannot use client-id quotas since we need to guarantee that one tenant's > quotas can never be used by another tenant. I suppose a multi-user cluster > where users trust each other could apply separate quotas for both clients > and users, but I am not sure if there is a usecase that can't be satisfied > with just client-id based quotas for this case. Do you have a usecase in > mind where you want to apply separate quotas for clients and users > simultaneously? > > > > > On Thu, Jun 9, 2016 at 9:40 PM, Jay Kreps <j...@confluent.io> wrote: > > > Super sorry to come in late on this one. Rajini, I had two quick > questions > > I think we should be able to answer: > > > > 1. Do client ids make sense in a world which has users? If not should > we > > unify them the way Hadoop did (without auth the user is a kind of best > > effort honor system identity). This came up in the discussion thread > > but I > > didn't really see a crisp answer. Basically, what is the definition of > > "client id" and what is the definition of "user" and how do the > concepts > > relate? > > 2. If both client ids and users are sensible distinct notions and we > > want to maintain both, why don't we just support quotas on both? If > they > > both make sense then you would have a reason to set quotas at both > > levels. > > Why have this "mode" that you set that swaps between only being able > to > > use > > one and the other? I should be able to set quotas at both levels. > Going > > forward the model we had discussed with quotas was potentially being > > able > > to set quotas for many things independently (say at the topic level), > > and I > > don't think it would make sense to extend this mode approach to those. > > > > -Jay > > > > On Wed, Jun 8, 2016 at 12:56 PM, Rajini Sivaram < > > rajinisiva...@googlemail.com> wrote: > > > > > I would like to initiate the vote for KIP-55. > > > > > > The KIP details are here: KIP-55: Secure quotas for authenticated users > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users > > > > > > > . > > > > > > The JIRA KAFKA-3492 < > https://issues.apache.org/jira/browse/KAFKA-3492 > > > >has > > > a draft PR here: https://github.com/apache/kafka/pull/1256. > > > > > > Thank you... > > > > > > Regards, > > > > > > Rajini > > > > > > > > > -- > Regards, > > Rajini >
