Sorry, for the confusion. I'm not sure my last email is clear enough either...
Consumers will have a Principal which may belong to a group. But consumer configuration also have a group.id, which controls how partitions are shared between consumers and how offsets are committed. I'm talking about those Groups. On Fri, Apr 24, 2015 at 11:33 AM, Gwen Shapira <gshap...@cloudera.com> wrote: > We are not talking about same Groups :) > > I meant, Groups of consumers (which KIP-11 lists as a separate > resource in the Privilege table) > > On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt > <pbrahmbh...@hortonworks.com> wrote: >> I see Groups as something we can add incrementally in the current model. >> The acls take principalType: name so groups can be represented as group: >> groupName. We are not managing group memberships anywhere in kafka and I >> don’t see the need to do so. >> >> So for a topic1 using the CLI an admin can add an acl to grant access to >> group:kafka-test-users. >> >> The authorizer implementation can have a plugin to map authenticated user >> to groups ( This is how hadoop and storm works). The plugin could be >> mapping user to linux/ldap/active directory groups but that is again upto >> the implementation. >> >> What we are offering is an interface that is extensible so these features >> can be added incrementally. I can add support for this in the first >> release but don’t necessarily see why this would be absolute necessity. >> >> Thanks >> Parth >> >> On 4/24/15, 11:00 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >> >>>Thanks. >>> >>>One more thing I'm missing in the KIP is details on the Group resource >>>(I think we discussed this and it was just not fully updated): >>> >>>* Does everyone have the privilege to create a new Group and use it to >>>consume from Topics he's already privileged on? >>>* Will the CLI tool be used to manage group membership too? >>>* Groups are kind of ephemeral, right? If all consumers in the group >>>disconnect the group is gone, AFAIK. Do we preserve the ACLs? Or do we >>>treat the new group as completely new resource? Can we create ACLs >>>before the group exists, in anticipation of it getting created? >>> >>>Its all small details, but it will be difficult to implement KIP-11 >>>without knowing the answers :) >>> >>>Gwen >>> >>> >>>On Fri, Apr 24, 2015 at 9:58 AM, Parth Brahmbhatt >>><pbrahmbh...@hortonworks.com> wrote: >>>> You are right, moved it to the default implementation section. >>>> >>>> Thanks >>>> Parth >>>> >>>> On 4/24/15, 9:52 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >>>> >>>>>Sample ACL JSON and Zookeeper is in public API, but I thought it is >>>>>part of DefaultAuthorizer (Since Sentry and Argus won't be using >>>>>Zookeeper). >>>>>Am I wrong? Or is it the KIP? >>>>> >>>>>On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt >>>>><pbrahmbh...@hortonworks.com> wrote: >>>>>> Thanks for clarifying Gwen, KIP updated. >>>>>> >>>>>> I tried to make the distinction by creating a section for all public >>>>>>APIs >>>>>> >>>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio >>>>>>n+ >>>>>>In >>>>>> terface#KIP-11-AuthorizationInterface-PublicInterfacesandclasses >>>>>> >>>>>> Let me know if you think there is a better way to reflect this. >>>>>> >>>>>> Thanks >>>>>> Parth >>>>>> >>>>>> On 4/24/15, 9:37 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >>>>>> >>>>>>>+1 (non-binding) >>>>>>> >>>>>>>Two nitpicks for the wiki: >>>>>>>* Heartbeat is probably a READ and not CLUSTER operation. I'm pretty >>>>>>>sure new consumers need it to be part of a consumer group. >>>>>>>* Can you clearly separate which parts are the API (common to every >>>>>>>Authorizer) and which parts are DefaultAuthorizer implementation? It >>>>>>>will make reviews and Authorizer implementations a bit easier to know >>>>>>>exactly which is which. >>>>>>> >>>>>>>Gwen >>>>>>> >>>>>>>On Fri, Apr 24, 2015 at 9:28 AM, Parth Brahmbhatt >>>>>>><pbrahmbh...@hortonworks.com> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I would like to open KIP-11 for voting. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Parth >>>>>>>> >>>>>>>> On 4/22/15, 1:56 PM, "Parth Brahmbhatt" >>>>>>>><pbrahmbh...@hortonworks.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>>Hi Jeff, >>>>>>>>> >>>>>>>>>Thanks a lot for the review. I think you have a valid point about >>>>>>>>>acls >>>>>>>>>being duplicated and the simplest solution would be to modify acls >>>>>>>>>class >>>>>>>>>so they hold a set of principals instead of single principal. i.e >>>>>>>>> >>>>>>>>><user_a,user_b> has <READ,WRITE,DESCRIBE> Permissions on <Topic1> >>>>>>>>>from >>>>>>>>><Host1, Host2, Host3>. >>>>>>>>> >>>>>>>>>I think the evaluation order only matters for the permissionType >>>>>>>>>which >>>>>>>>>is >>>>>>>>>Deny acls should be evaluated before allow acls. To give you an >>>>>>>>>example >>>>>>>>>suppose we have following acls >>>>>>>>> >>>>>>>>>acl1 -> user1 is allowed to READ from all hosts. >>>>>>>>>acl2 -> host1 is allowed to READ regardless of who is the user. >>>>>>>>>acl3 -> host2 is allowed to READ regardless of who is the user. >>>>>>>>> >>>>>>>>>acl4 -> user1 is denied to READ from host1. >>>>>>>>> >>>>>>>>>As stated in the KIP we first evaluate DENY so if user1 tries to >>>>>>>>>access >>>>>>>>>from host1 he will be denied(acl4), even though both user1 and host1 >>>>>>>>>has >>>>>>>>>acl’s for allow with wildcards (acl1, acl2). >>>>>>>>>If user1 tried to READ from host2 , the action will be allowed and >>>>>>>>>it >>>>>>>>>does >>>>>>>>>not matter if we match acl3 or acl1 so I don’t think the evaluation >>>>>>>>>order >>>>>>>>>matters here. >>>>>>>>> >>>>>>>>>“Will people actually use hosts with users?” I really don’t know but >>>>>>>>>given >>>>>>>>>ACl’s are part of our Public APIs I thought it is better to try and >>>>>>>>>cover >>>>>>>>>more use cases. If others think this extra complexity is not worth >>>>>>>>>the >>>>>>>>>value its adding please raise your concerns so we can discuss if it >>>>>>>>>should >>>>>>>>>be removed from the acl structure. Note that even in absence of >>>>>>>>>hosts >>>>>>>>>from >>>>>>>>>ACL users will still be able to whitelist/blacklist host as long as >>>>>>>>>we >>>>>>>>>start supporting principalType = “host”, easy to add and can be an >>>>>>>>>incremental improvement. They will however loose the ability to >>>>>>>>>restrict >>>>>>>>>access to users just from a set of hosts. >>>>>>>>> >>>>>>>>>We agreed to offer a CLI to overcome the JSON acl config >>>>>>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza >>>>>>>>>ti >>>>>>>>>on >>>>>>>>>+I >>>>>>>>>n >>>>>>>>>terface#KIP-11-AuthorizationInterface-AclManagement(CLI). I still >>>>>>>>>like >>>>>>>>>Jsons but that probably has something to do with me being a >>>>>>>>>developer >>>>>>>>>:-). >>>>>>>>> >>>>>>>>>Thanks >>>>>>>>>Parth >>>>>>>>> >>>>>>>>>On 4/22/15, 11:38 AM, "Jeff Holoman" <jholo...@cloudera.com> wrote: >>>>>>>>> >>>>>>>>>>Parth, >>>>>>>>>> >>>>>>>>>>This is a long thread, so trying to keep up here, sorry if this has >>>>>>>>>>been >>>>>>>>>>covered before. First, great job on the KIP proposal and work so >>>>>>>>>>far. >>>>>>>>>> >>>>>>>>>>Are we sure that we want to tie host level access to a given user? >>>>>>>>>>My >>>>>>>>>>understanding is that the ACL will be (omitting some fields) >>>>>>>>>> >>>>>>>>>>user_a, host1, host2, host3 >>>>>>>>>>user_b, host1, host2, host3 >>>>>>>>>> >>>>>>>>>>So there would potentially be a lot of redundancy in the configs. >>>>>>>>>>Does >>>>>>>>>>it >>>>>>>>>>make sense to have hosts be at the same level as principal in the >>>>>>>>>>hierarchy? This way you could just blanket the allowed / denied >>>>>>>>>>hosts >>>>>>>>>>and >>>>>>>>>>only have to worry about the users. So if you follow this, then >>>>>>>>>> >>>>>>>>>>we can wildcard the user so we can have a separate list of just >>>>>>>>>>host-based >>>>>>>>>>access. What's the order that the perms would be evaluated if a >>>>>>>>>>there >>>>>>>>>>was >>>>>>>>>>more than one match on a principal ? >>>>>>>>>> >>>>>>>>>>Is the thought that there wouldn't usually be much overlap on >>>>>>>>>>hosts? >>>>>>>>>>I >>>>>>>>>>guess I can imagine a scenario where I want to offline/online >>>>>>>>>>access >>>>>>>>>>to a >>>>>>>>>>particular hosts or set of hosts and if there was overlap, I'm >>>>>>>>>>doing >>>>>>>>>>a >>>>>>>>>>bunch of alter commands for just a single host. Maybe this is too >>>>>>>>>>contrived >>>>>>>>>>an example? >>>>>>>>>> >>>>>>>>>>I agree that having this level of granularity gives flexibility >>>>>>>>>>but I >>>>>>>>>>wonder if people will actually use it and not just * the hosts for >>>>>>>>>>a >>>>>>>>>>given >>>>>>>>>>user and create separate "global" list as i mentioned above? >>>>>>>>>> >>>>>>>>>>The only other system I know of that ties users with hosts for >>>>>>>>>>access >>>>>>>>>>is >>>>>>>>>>MySql and I don't love that model. Companies usually standardize on >>>>>>>>>>group >>>>>>>>>>authorization anyway, are we complicating that issue with the >>>>>>>>>>inclusion >>>>>>>>>>of >>>>>>>>>>hosts attached to users? Additionally I worry about the debt of big >>>>>>>>>>JSON >>>>>>>>>>configs in the first place, most non-developers find them >>>>>>>>>>non-intuitive >>>>>>>>>>already, so anything to ease this I think would be beneficial. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Thanks >>>>>>>>>> >>>>>>>>>>Jeff >>>>>>>>>> >>>>>>>>>>On Wed, Apr 22, 2015 at 2:22 PM, Parth Brahmbhatt < >>>>>>>>>>pbrahmbh...@hortonworks.com> wrote: >>>>>>>>>> >>>>>>>>>>> Sorry I missed your last questions. I am +0 on adding ―host >>>>>>>>>>>option >>>>>>>>>>>for >>>>>>>>>>> ―list, we could add it for symmetry. Again if this is only a CLI >>>>>>>>>>>change >>>>>>>>>>>it >>>>>>>>>>> can be added later if you mean adding this in authorizer >>>>>>>>>>>interface >>>>>>>>>>>then >>>>>>>>>>>we >>>>>>>>>>> should make a decision now. >>>>>>>>>>> >>>>>>>>>>> Given a choice I would like to actually keep only one option >>>>>>>>>>>which >>>>>>>>>>>is >>>>>>>>>>> resource based get (remove even the get based on principal). I >>>>>>>>>>>see >>>>>>>>>>>those >>>>>>>>>>> (getAcl for principal or host) as special filtering case which >>>>>>>>>>>can >>>>>>>>>>>easily >>>>>>>>>>> be achieved by a third party tool by doing "list all topics" and >>>>>>>>>>>calling >>>>>>>>>>> getAcls for each topic and applying filtering logic on that. I >>>>>>>>>>>really >>>>>>>>>>> don’t see the need to make those first class citizens of the >>>>>>>>>>>authorizer >>>>>>>>>>> interface given these kind of queries will be issued outside of >>>>>>>>>>>broker >>>>>>>>>>>JVM >>>>>>>>>>> so they will not benefit from the caching and because the storage >>>>>>>>>>>will >>>>>>>>>>>be >>>>>>>>>>> indexed on resource both these options even as a first class API >>>>>>>>>>>will >>>>>>>>>>>just >>>>>>>>>>> scan all topic acls and apply filtering logic. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Parth >>>>>>>>>>> >>>>>>>>>>> On 4/22/15, 11:08 AM, "Parth Brahmbhatt" >>>>>>>>>>><pbrahmbh...@hortonworks.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >Please see all the available options here >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authori >>>>>>>>>>>za >>>>>>>>>>>ti >>>>>>>>>>>on >>>>>>>>>>>+ >>>>>>>>>>>I >>>>>>>>>>> >nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I >>>>>>>>>>>think >>>>>>>>>>>it >>>>>>>>>>> >covers both hosts and operations and allows to specify a list >>>>>>>>>>>for >>>>>>>>>>>both. >>>>>>>>>>> > >>>>>>>>>>> >Thanks >>>>>>>>>>> >Parth >>>>>>>>>>> > >>>>>>>>>>> >From: Tom Graves >>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com>> >>>>>>>>>>> >Reply-To: Tom Graves >>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com>> >>>>>>>>>>> >Date: Wednesday, April 22, 2015 at 11:02 AM >>>>>>>>>>> >To: Parth Brahmbhatt >>>>>>>>>>> >>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>>>>>>>>>>>, >>>>>>>>>>> >"dev@kafka.apache.org<mailto:dev@kafka.apache.org>" >>>>>>>>>>> ><dev@kafka.apache.org<mailto:dev@kafka.apache.org>> >>>>>>>>>>> >Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka >>>>>>>>>>>security >>>>>>>>>>> > >>>>>>>>>>> >Thanks for the explanations Parth. >>>>>>>>>>> > >>>>>>>>>>> >On the configs questions, the way I see it is its more likely to >>>>>>>>>>> >accidentally give everyone access, especially since you have to >>>>>>>>>>>run >>>>>>>>>>>a >>>>>>>>>>> >separate command to change the acls. If there was some config >>>>>>>>>>>for >>>>>>>>>>> >defaults, a cluster admin could change that to be nobody or >>>>>>>>>>>certain >>>>>>>>>>>set >>>>>>>>>>> >of users, then grant others permissions. This would also remove >>>>>>>>>>>the >>>>>>>>>>>race >>>>>>>>>>> >between commands. This is something you can always add later >>>>>>>>>>>though >>>>>>>>>>>if >>>>>>>>>>> >people request it. >>>>>>>>>>> > >>>>>>>>>>> >So in kafka-acl.sh how do I actually tell it what the operation >>>>>>>>>>>is? >>>>>>>>>>> >kafka-acl.sh --topic testtopic --add --grandprincipal >>>>>>>>>>>user:joe,user:kate >>>>>>>>>>> > >>>>>>>>>>> >where does READ, WRITE, etc go? Can specify as a list so I >>>>>>>>>>>don't >>>>>>>>>>>have >>>>>>>>>>>to >>>>>>>>>>> >run this a bunch of times for each. >>>>>>>>>>> > >>>>>>>>>>> >Do you want to have a --host option for --list so that admins >>>>>>>>>>>could >>>>>>>>>>>see >>>>>>>>>>> >what acls apply to specific host(s)? >>>>>>>>>>> > >>>>>>>>>>> >Tom >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> >On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt >>>>>>>>>>> >>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>>>>>>>>>>wrote: >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> >FYI, I have modified the KIP to include group as resource. In >>>>>>>>>>>order >>>>>>>>>>>to >>>>>>>>>>> >access “joinGroup” and “commitOFfset” APIs the user will need a >>>>>>>>>>>read >>>>>>>>>>> >permission on topic and WRITE permission on group. >>>>>>>>>>> > >>>>>>>>>>> >I plan to open a VOTE thread by noon if there are no more >>>>>>>>>>>concerns. >>>>>>>>>>> > >>>>>>>>>>> >Thanks >>>>>>>>>>> >Parth >>>>>>>>>>> > >>>>>>>>>>> >On 4/22/15, 9:03 AM, "Tom Graves" >>>>>>>>>>> >>>>>>>>>>>><tgraves...@yahoo.com.INVALID<mailto:tgraves...@yahoo.com.INVALID >>>>>>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> > >>>>>>>>>>> >>Hey everyone, >>>>>>>>>>> >>Sorry to jump in on the conversation so late. I'm new to Kafka. >>>>>>>>>>>I'll >>>>>>>>>>> >>apologize in advance if you have already covered some of my >>>>>>>>>>>questions. I >>>>>>>>>>> >>read through the wiki and had some comments and questions. >>>>>>>>>>> >>1) public enum Operation needs EDIT changed to ALTER >>>>>>>>>>> > >>>>>>>>>>> >> Done. >>>>>>>>>>> > >>>>>>>>>>> >>2) Does the Authorizer class need a setAcls? Rather then just >>>>>>>>>>>add >>>>>>>>>>>to >>>>>>>>>>>be >>>>>>>>>>> >>able to set to explicit list and overwrite what was there? I >>>>>>>>>>>see >>>>>>>>>>>the >>>>>>>>>>> >>kafka-acl.sh lists a removeall so I guess you could do >>>>>>>>>>>removeall >>>>>>>>>>>and >>>>>>>>>>>then >>>>>>>>>>> >>add. I also don't see a removeall in the Authorizer class, is >>>>>>>>>>>it >>>>>>>>>>>going >>>>>>>>>>> >>to loop through them all to remove each one? >>>>>>>>>>> > >>>>>>>>>>> > There is an overloaded version of removeAcls in the >>>>>>>>>>>interface >>>>>>>>>>>that >>>>>>>>>>> >takes >>>>>>>>>>> >in resource as the only input and as described in the javadoc >>>>>>>>>>>all >>>>>>>>>>>the >>>>>>>>>>>acls >>>>>>>>>>> >attached to that resource will be deleted. To cover the setAcl >>>>>>>>>>>use >>>>>>>>>>>case >>>>>>>>>>> >the caller can first call remove and then add. >>>>>>>>>>> > >>>>>>>>>>> >>3) Can someone tell me what the use case to do acls based on >>>>>>>>>>>the >>>>>>>>>>>hosts? >>>>>>>>>>> >>I can see some possibilities just wondering if we can concrete >>>>>>>>>>>ones >>>>>>>>>>>where >>>>>>>>>>> >>one user is allowed from one host but not another. >>>>>>>>>>> > >>>>>>>>>>> > I am not sure if I understand the question given the use >>>>>>>>>>>case >>>>>>>>>>>you >>>>>>>>>>> >described in your question is what we are trying to cover with >>>>>>>>>>>use >>>>>>>>>>>of >>>>>>>>>>> >hosts in Acl. There are some additional use cases like “allow >>>>>>>>>>>access >>>>>>>>>>>to >>>>>>>>>>> >any user from host1,host2” but I think primarily it gives the >>>>>>>>>>>admins >>>>>>>>>>>the >>>>>>>>>>> >ability to define acls at a more granular level. >>>>>>>>>>> > >>>>>>>>>>> >>4) I'm a bit unclear how the "resource" works in the Authorizer >>>>>>>>>>>class. >>>>>>>>>>> >>From what I see we have 2 resources - topics and cluster. If I >>>>>>>>>>>want >>>>>>>>>>>to >>>>>>>>>>> >>add an acl to allow "joe" to CREATE for the cluster then I call >>>>>>>>>>>addAcls >>>>>>>>>>> >>with Acl("user: joe", ALLOW, Set(*), Set(CREATE)) and >>>>>>>>>>>"cluster"? >>>>>>>>>>>What >>>>>>>>>>> >>if I want to call addAcls for DESCRIBE on a topic? Is the >>>>>>>>>>>resource >>>>>>>>>>>then >>>>>>>>>>> >>"topic" or is it the topic name? >>>>>>>>>>> > >>>>>>>>>>> > We now have 3 resources(added group), please see the updated >>>>>>>>>>>doc. >>>>>>>>>>>The >>>>>>>>>>> >CREATE acl that you described is correct. For any topic >>>>>>>>>>>operation >>>>>>>>>>>you >>>>>>>>>>> >should use topic name as the resource name and for group the >>>>>>>>>>>user >>>>>>>>>>>will >>>>>>>>>>> >provide groupId as resource name. >>>>>>>>>>> > >>>>>>>>>>> >>5) reassigning partitions is a CLUSTER_ACTION or superuser? >>>>>>>>>>>Its >>>>>>>>>>>not >>>>>>>>>>> >>totally clear to me the differences between these. what about >>>>>>>>>>>increasing >>>>>>>>>>> >># of partitions? >>>>>>>>>>> > >>>>>>>>>>> > I see this as an alter topic operation so it is at topic >>>>>>>>>>>level >>>>>>>>>>>and >>>>>>>>>>>the >>>>>>>>>>> >user must have alter permissions on topic. >>>>>>>>>>> > >>>>>>>>>>> >>6) groups are mentioned, are we supporting right away or is >>>>>>>>>>>that >>>>>>>>>>>a >>>>>>>>>>>follow >>>>>>>>>>> >>on item? (is there going to be a kafka.supergroups) >>>>>>>>>>> > >>>>>>>>>>> > I think it can be a separate jira just for braking down the >>>>>>>>>>>code >>>>>>>>>>> >review >>>>>>>>>>> >in smaller chunk. We will support it in first version but I >>>>>>>>>>>think >>>>>>>>>>>if >>>>>>>>>>>we >>>>>>>>>>> >can not do it for any reason that should not block a release >>>>>>>>>>>with >>>>>>>>>>>all >>>>>>>>>>>the >>>>>>>>>>> >other authZ work. We made deliberate design choices (like >>>>>>>>>>>introducing >>>>>>>>>>>a >>>>>>>>>>> >principalType in KafkaPrinciapl) to allow supporting groups as >>>>>>>>>>>an >>>>>>>>>>> >incremental change. >>>>>>>>>>> > >>>>>>>>>>> >>7) Are there config options for setting acls when I create my >>>>>>>>>>>topic? >>>>>>>>>>>Or >>>>>>>>>>> >>do I have to create my topic and then run the kafka-acl.sh >>>>>>>>>>>script >>>>>>>>>>>to >>>>>>>>>>>set >>>>>>>>>>> >>them? Although its very small, there would be possible race >>>>>>>>>>>there >>>>>>>>>>>that >>>>>>>>>>> >>someone could start producing to topic before acls are set. >>>>>>>>>>> > >>>>>>>>>>> > We discussed this yesterday and we agreed to go with >>>>>>>>>>>kafka-acl.sh. >>>>>>>>>>>Yes >>>>>>>>>>> >there is a very very small window of vulnerability but I think >>>>>>>>>>>that >>>>>>>>>>>really >>>>>>>>>>> >does not warrant to change the decision in this case. >>>>>>>>>>> > >>>>>>>>>>> >>8) are there configs for cluster level acl defaults? Or does >>>>>>>>>>>it >>>>>>>>>>>default >>>>>>>>>>> >>to superusers on bringing up new cluster and you have to modify >>>>>>>>>>>with >>>>>>>>>>>cli. >>>>>>>>>>> >>thanks,Tom >>>>>>>>>>> > >>>>>>>>>>> > No defaults, the default is superusers will have full >>>>>>>>>>>access. >>>>>>>>>>>I >>>>>>>>>>>don’t >>>>>>>>>>> >think making assumptions about ones security requirement should >>>>>>>>>>>be >>>>>>>>>>>our >>>>>>>>>>> >burden. >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> >> >>>>>>>>>>> >> On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt >>>>>>>>>>> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com> >>>>>>>>>>>>>> >>>>>>>>>>>wrote: >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> I have added the notes to KIP-11 Open question sections. >>>>>>>>>>> >> >>>>>>>>>>> >>Thanks >>>>>>>>>>> >>Parth >>>>>>>>>>> >> >>>>>>>>>>> >>On 4/21/15, 4:49 PM, "Gwen Shapira" >>>>>>>>>>> >><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> wrote: >>>>>>>>>>> >> >>>>>>>>>>> >>>Adding my notes from today's call to the thread: >>>>>>>>>>> >>> >>>>>>>>>>> >>>** Deny or Allow all by default? We will add a configuration >>>>>>>>>>>to >>>>>>>>>>> >>>control this. The configuration will default to “allow” for >>>>>>>>>>>backward >>>>>>>>>>> >>>compatibility. Security admins can set it to "deny" >>>>>>>>>>> >>> >>>>>>>>>>> >>>** Storing ACLs for default authorizers: We'll store them in >>>>>>>>>>>ZK. >>>>>>>>>>>We'll >>>>>>>>>>> >>>support pointing the authorizer to any ZK. >>>>>>>>>>> >>>The use of ZK will be internal to the default authorizer. >>>>>>>>>>>Authorizer >>>>>>>>>>> >>>reads ACLs from cache every hour. We proposed having mechanism >>>>>>>>>>> >>>(possibly via new ZK node) to tell broker to refresh the cache >>>>>>>>>>> >>>immediately. >>>>>>>>>>> >>> >>>>>>>>>>> >>>** Support deny as permission type - we agreed to keep this. >>>>>>>>>>> >>> >>>>>>>>>>> >>>** Mapping operations to API: We may need to add Group as a >>>>>>>>>>>resource, >>>>>>>>>>> >>>with JoinGroup and OffsetCommit require privilege on the >>>>>>>>>>>consumer >>>>>>>>>>> >>>group. >>>>>>>>>>> >>>This can be something we pass now and authorizers can support >>>>>>>>>>>in >>>>>>>>>>> >>>future. - Jay will write specifics to the mailing list >>>>>>>>>>>discussion. >>>>>>>>>>> >>> >>>>>>>>>>> >>>On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps >>>>>>>>>>> >>><jay.kr...@gmail.com<mailto:jay.kr...@gmail.com>> wrote: >>>>>>>>>>> >>>> Following up on the KIP discussion. Two options for >>>>>>>>>>>authorizing >>>>>>>>>>> >>>>consumers >>>>>>>>>>> >>>> to read topic "t" as part of group "g": >>>>>>>>>>> >>>> 1. READ permission on resource /topic/t >>>>>>>>>>> >>>> 2. READ permission on resource /topic/t AND WRITE permission >>>>>>>>>>>on >>>>>>>>>>> >>>>/group/g >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> The advantage of (1) is that it is simpler. The disadvantage >>>>>>>>>>>is >>>>>>>>>>>that >>>>>>>>>>> >>>>any >>>>>>>>>>> >>>> member of any group that reads from t can commit offsets as >>>>>>>>>>>any >>>>>>>>>>>other >>>>>>>>>>> >>>> member of a different group. This doesn't effect data >>>>>>>>>>>security >>>>>>>>>>>(who >>>>>>>>>>> >>>>can >>>>>>>>>>> >>>> access what) but it is a bit of a management issue--a >>>>>>>>>>>malicious >>>>>>>>>>>person >>>>>>>>>>> >>>>can >>>>>>>>>>> >>>> cause data loss or duplicates for another consumer by >>>>>>>>>>>committing >>>>>>>>>>> >>>>offset. >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> I think I favor (2) but it's worth it to think it through. >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> -Jay >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt < >>>>>>>>>>> >>>> >>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>>>>>>>>>> >>>>wrote: >>>>>>>>>>> >>>> >>>>>>>>>>> >>>>> Hey Jun, >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> Yes and we support wild cards for all acl entities >>>>>>>>>>>principal, >>>>>>>>>>>hosts >>>>>>>>>>> >>>>>and >>>>>>>>>>> >>>>> operation. >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> Thanks >>>>>>>>>>> >>>>> Parth >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> On 4/21/15, 9:06 AM, "Jun Rao" >>>>>>>>>>> >>>>><j...@confluent.io<mailto:j...@confluent.io>> wrote: >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> >Harsha, Parth, >>>>>>>>>>> >>>>> > >>>>>>>>>>> >>>>> >Thanks for the clarification. This makes sense. Perhaps we >>>>>>>>>>>can >>>>>>>>>>> >>>>>clarify the >>>>>>>>>>> >>>>> >meaning of those rules in the wiki. >>>>>>>>>>> >>>>> > >>>>>>>>>>> >>>>> >Related to this, it seems that we need to support wildcard >>>>>>>>>>>in >>>>>>>>>>> >>>>>cli/request >>>>>>>>>>> >>>>> >protocol for topics? >>>>>>>>>>> >>>>> > >>>>>>>>>>> >>>>> >Jun >>>>>>>>>>> >>>>> > >>>>>>>>>>> >>>>> >On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt < >>>>>>>>>>> >>>>> >>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>>>>>>>>>> >>>>>wrote: >>>>>>>>>>> >>>>> > >>>>>>>>>>> >>>>> >> The iptables on unix supports the DENY operator, not >>>>>>>>>>>that >>>>>>>>>>>it >>>>>>>>>>> >>>>>should >>>>>>>>>>> >>>>> >> matter. The deny operator can also be used to specify >>>>>>>>>>>³allow >>>>>>>>>>>user1 >>>>>>>>>>> >>>>>to >>>>>>>>>>> >>>>> >>READ >>>>>>>>>>> >>>>> >> from topic1 from all hosts but host1,host2². Again we >>>>>>>>>>>could >>>>>>>>>>>add a >>>>>>>>>>> >>>>>host >>>>>>>>>>> >>>>> >> group semantic and extra complexity around that, not >>>>>>>>>>>sure >>>>>>>>>>>if >>>>>>>>>>>its >>>>>>>>>>> >>>>>worth >>>>>>>>>>> >>>>> >>it. >>>>>>>>>>> >>>>> >> In addition with DENY operator you are now not forced to >>>>>>>>>>>create a >>>>>>>>>>> >>>>> >>special >>>>>>>>>>> >>>>> >> group just to support the authorization use case. I am >>>>>>>>>>>not >>>>>>>>>>> >>>>>convinced >>>>>>>>>>> >>>>> >>that >>>>>>>>>>> >>>>> >> the operator it self is really all that confusing. There >>>>>>>>>>>are 3 >>>>>>>>>>> >>>>>practical >>>>>>>>>>> >>>>> >> use cases: >>>>>>>>>>> >>>>> >> - Resource with no acl what so ever -> allow access to >>>>>>>>>>>everyone ( >>>>>>>>>>> >>>>>just >>>>>>>>>>> >>>>> >>for >>>>>>>>>>> >>>>> >> backward compatibility, I would much rather fail close >>>>>>>>>>>and >>>>>>>>>>>force >>>>>>>>>>> >>>>>users >>>>>>>>>>> >>>>> >>to >>>>>>>>>>> >>>>> >> explicitly grant acls that allows access to all users.) >>>>>>>>>>> >>>>> >> - Resource with some acl attached -> only users that >>>>>>>>>>>have >>>>>>>>>>>a >>>>>>>>>>> >>>>>matching >>>>>>>>>>> >>>>> >>allow >>>>>>>>>>> >>>>> >> acl are allowed (i.e. ³allow READ access to topic1 to >>>>>>>>>>>user1 >>>>>>>>>>>from >>>>>>>>>>> >>>>>all >>>>>>>>>>> >>>>> >> hosts², only user1 has READ access and no other user has >>>>>>>>>>>access of >>>>>>>>>>> >>>>>any >>>>>>>>>>> >>>>> >> kind) >>>>>>>>>>> >>>>> >> - Resource with some allow and some deny acl attached -> >>>>>>>>>>>users >>>>>>>>>>>are >>>>>>>>>>> >>>>> >>allowed >>>>>>>>>>> >>>>> >> to perform operation only when they satisfy allow acl >>>>>>>>>>>and >>>>>>>>>>>do >>>>>>>>>>>not >>>>>>>>>>> >>>>>have >>>>>>>>>>> >>>>> >> conflicting deny acl. Users that have no acl(allow or >>>>>>>>>>>deny) >>>>>>>>>>>will >>>>>>>>>>> >>>>>still >>>>>>>>>>> >>>>> >>not >>>>>>>>>>> >>>>> >> have any access. (i.e. ³allow READ access to topic1 to >>>>>>>>>>>user1 >>>>>>>>>>>from >>>>>>>>>>> >>>>>all >>>>>>>>>>> >>>>> >> hosts except host1 and host², only user1 has access but >>>>>>>>>>>not >>>>>>>>>>>from >>>>>>>>>>> >>>>>host1 >>>>>>>>>>> >>>>> >>an >>>>>>>>>>> >>>>> >> host2) >>>>>>>>>>> >>>>> >> >>>>>>>>>>> >>>>> >> I think we need to make a decision on deny primarily >>>>>>>>>>>because >>>>>>>>>>>with >>>>>>>>>>> >>>>> >> introduction of acl management API, Acl is now a public >>>>>>>>>>>class >>>>>>>>>>>that >>>>>>>>>>> >>>>>will >>>>>>>>>>> >>>>> >>be >>>>>>>>>>> >>>>> >> used by Ranger/Santry and other authroization providers. >>>>>>>>>>>In >>>>>>>>>>> >>>>>Current >>>>>>>>>>> >>>>> >>design >>>>>>>>>>> >>>>> >> the acl has a permissionType enum field with possible >>>>>>>>>>>values >>>>>>>>>>>of >>>>>>>>>>> >>>>>Allow >>>>>>>>>>> >>>>> >>and >>>>>>>>>>> >>>>> >> Deny. If we chose to remove deny we can assume all acls >>>>>>>>>>>to >>>>>>>>>>>be >>>>>>>>>>>of >>>>>>>>>>> >>>>>allow >>>>>>>>>>> >>>>> >> type and remove the permissionType field completely. >>>>>>>>>>> >>>>> >> >>>>>>>>>>> >>>>> >> Thanks >>>>>>>>>>> >>>>> >> Parth >>>>>>>>>>> >>>>> >> >>>>>>>>>>> >>>>> >> On 4/20/15, 6:12 PM, "Gwen Shapira" >>>>>>>>>>> >>>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> wrote: >>>>>>>>>>> >>>>> >> >>>>>>>>>>> >>>>> >> >I think thats how its done in pretty much any system I >>>>>>>>>>>can >>>>>>>>>>>think >>>>>>>>>>> >>>>>of. >>>>>>>>>>> >>>>> >> > >>>>>>>>>>> >>>>> >> >>>>>>>>>>> >>>>> >> >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>-- >>>>>>>>>>Jeff Holoman >>>>>>>>>>Systems Engineer >>>>>>>>> >>>>>>>> >>>>>> >>>> >>
