I also wanted to send ping to all he committers. This voting thread has been open for > 1 week and has 2 non-bindng +1s. I would appreciate if the committers raised their concerns or casted their votes.
Thanks Parth On 4/30/15, 9:52 AM, "Parth Brahmbhatt" <pbrahmbh...@hortonworks.com> wrote: >Hi Joe, Thanks for taking the time to review. > >* All the open issues already have a resolution , I can open a jira for >each one and add the resolution to it and resolve them immediately if you >want this for tracking purposes. >* We will update system tests to verify that the code works. We have >thorough unit tests fr all the new code except for modifications made to >KafkaAPI as that has way too many dependencies to be mocked which I guess >is the reason for no existing unit tests. >* I don’t know if I completely understand the concern. We have talked with >Ranger team (Don Bosco Drai) so we at least have one custom authorizer >implementation that has approved this design and they will be able to >inject their authorization framework with current interfaces. Do you see >any issue with the design which will prevent anyone from providing a >custom implementation? >* Did not understand the concern around wire protocol, we are adding >AuthorizationException to indicate that an operation was not authorized. > >Thanks >Parth > >On 4/30/15, 5:59 AM, "Jun Rao" <j...@confluent.io> wrote: > >>Joe, >> >>Could you elaborate on why we should not store JSON in ZK? So far, all >>existing ZK data are in JSON. >> >>Thanks, >> >>Jun >> >>On Thu, Apr 30, 2015 at 2:06 AM, Joe Stein <joe.st...@stealth.ly> wrote: >> >>> Hi, sorry I am coming in late to chime back in on this thread and >>>haven't >>> been able to make the KIP hangouts the last few weeks. Sorry if any of >>>this >>> was brought up already or I missed it. >>> >>> I read through the KIP and the threads) and a couple of things jumped >>>out. >>> >>> >>> - Can we break out the open issues in JIRA (maybe during the >>>hangout) >>> that are in the KIP and resolve/flesh those out more? >>> >>> >>> >>> - I don't see any updates with the systems test or how wecan know >>>the >>> code works. >>> >>> >>> >>> - We need some implementation/example/sample that we know can work >>>in >>> all different existing entitlement servers and not just ones that >>>run in >>> types of data centers too. I am not saying we should support >>>everything >>> but >>> if someone had to implement >>> https://docs.oracle.com/cd/E19225-01/820-6551/bzafm/index.html with >>> Kafka it has to work for them out of the box. >>> >>> >>> >>> - We should shy away from storing JSON in Zookeeper. Lets store >>>bytes in >>> Storage. >>> >>> >>> >>> - We should spend some time thinking through exceptions in the wire >>> protocol maybe as part of this so it can keep moving forward. >>> >>> >>> ~ Joe Stein >>> >>> On Tue, Apr 28, 2015 at 3:33 AM, Sun, Dapeng <dapeng....@intel.com> >>>wrote: >>> >>> > Thank you for your reply, Gwen. >>> > >>> > >1. Complex rule systems can be difficult to reason about and >>>terefore >>> > end up being less secure. The rule "Deny always wins" is very easy to >>> grasp. >>> > Yes, I'm agreed with your point: we should not make the rule complex. >>> > >>> > >2. We currently don't have any mechanism for specifying IP ranges >>>(or >>> host >>> > >ranges) at all. I think its a pretty significant deficiency, but it >>>does >>> > mean that we don't need to worry about the issue of blocking a large >>> range >>> > while unblocking few servers in the range. >>> > Support ranges sounds reasonable. If this feature will be in >>>development >>> > plan, I also don't think we can put "the best matching acl" and " >>>Support >>> > ip ranges" together. >>> > >>> > >We have a call tomorrow (Tuesday, April 28) at 3pm PST - to discuss >>>this >>> > and other outstanding design issues (not all related to security). If >>>you >>> > are interested in joining - let me know and I'll forward you the >>>invite. >>> > Thank you, Gwen. I have the invite and I should be at home at that >>>time. >>> > But due to network issue, I may can't join the meeting smoothly. >>> > >>> > Regards >>> > Dapeng >>> > >>> > -----Original Message----- >>> > From: Gwen Shapira [mailto:gshap...@cloudera.com] >>> > Sent: Tuesday, April 28, 2015 1:31 PM >>> > To: dev@kafka.apache.org >>> > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security >>> > >>> > While I see the advantage of being able to say something like: "deny >>>user >>> > X from hosts h1...h200" also "allow user X from host h189", there are >>>two >>> > issues here: >>> > >>> > 1. Complex rule systems can be difficult to reason about and >>>therefore >>> end >>> > up being less secure. The rule "Deny always wins" is very easy to >>>grasp. >>> > >>> > 2. We currently don't have any mechanism for specifying IP ranges (or >>> host >>> > ranges) at all. I think its a pretty significant deficiency, but it >>>does >>> > mean that we don't need to worry about the issue of blocking a large >>> range >>> > while unblocking few servers in the range. >>> > >>> > Gwen >>> > >>> > P.S >>> > We have a call tomorrow (Tuesday, April 28) at 3pm PST - to discuss >>>this >>> > and other outstanding design issues (not all related to security). If >>>you >>> > are interested in joining - let me know and I'll forward you the >>>invite. >>> > >>> > Gwen >>> > >>> > On Mon, Apr 27, 2015 at 10:15 PM, Sun, Dapeng <dapeng....@intel.com> >>> > wrote: >>> > >>> > > Attach the image. >>> > > >>> > > >>>https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-ac >>> > > l1.png >>> > > >>> > > Regards >>> > > Dapeng >>> > > >>> > > From: Sun, Dapeng [mailto:dapeng....@intel.com] >>> > > Sent: Tuesday, April 28, 2015 11:44 AM >>> > > To: dev@kafka.apache.org >>> > > Subject: RE: [VOTE] KIP-11- Authorization design for kafka security >>> > > >>> > > >>> > > Thank you for your rapid reply, Parth. >>> > > >>> > > >>> > > >>> > > >* I think the wiki already describes the precedence order as Deny >>> > > >taking >>> > > precedence over allow when conflicting acls are found >>> > > >>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizati >>> > > on+In >>> > > >>> > > >terface#KIP-11-AuthorizationInterface-PermissionType >>> > > >>> > > Got it, thank you. >>> > > >>> > > >>> > > >>> > > >* In the first version that I am currently writing there is no >>>group >>> > > support. Even when we add it I don't see the need to add a >>>precedence >>> > > for evaluation. it does not matter which principal matches as long >>>as >>> > > >>> > > > we have a match. >>> > > >>> > > >>> > > >>> > > About this part, I think we should choose the best matching acl for >>> > > authorization, no matter we support group or not. >>> > > >>> > > >>> > > >>> > > For the case >>> > > >>> > > [cid:image001.png@01D08197.E94BD410] >>> > > >>> > > >>>https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-ac >>> > > l1.png >>> > > >>> > > >>> > > >>> > > if 2 Acls are defined, one that deny an operation from all hosts >>>and >>> > > one that allows the operation from host1, the operation from host1 >>> > > will be denied or allowed? >>> > > >>> > > According wiki "Deny will take precedence over Allow in competing >>> > > acls.", it seems acl_1 will win the competition, but customers' >>> > > intention may be "allow". >>> > > >>> > > I think "deny always take precedence over Allow" is okay, but >>>"host1 >>> > > -> user1" > "host1 " > "default" may make sense. >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >* Acl storage is indexed by resource right now because that is the >>> > > primary lookup id for all authorize operations. Given acls are >>>cached >>> > > I don't see the need to optimized the storage layer any further for >>> > lookup. >>> > > >>> > > >* The reason why we have acl with multi everything is to reduce >>> > > redundancy in acl storage. I am not sure how will we be able to >>>reduce >>> > > redundancy if we divide it by using one principal,one host, one >>> > operation. >>> > > >>> > > >>> > > >>> > > Yes, I'm also agreed with "Acl storage should be indexed by >>>resource". >>> > > Under resource index, it may be better to add index such as hosts >>>and >>> > > principals. One option may be one principal, one host, one >>>operation. >>> > > Just give your these scenarios for considering. >>> > > >>> > > >>> > > >>> > > For the case defined in wiki: >>> > > >>> > > Acl_1 -> {"user:bob", "user:*"} is allowed to READ from all hosts. >>> > > >>> > > Acl_2 -> {"user:bob"} is denied to READ from host1 >>> > > >>> > > Acl_3 -> {"user:alice", "group:kafka-devs"} is allowed to READ and >>> > > WRITE from {host1, host2}. >>> > > >>> > > >>> > > >>> > > For acl_3, if we want to remove alice's WRITE from {host1,host2} >>>and >>> > > remove alice's READ from host1, user may have following ways to >>> achieve: >>> > > >>> > > >>> > > >>> > > 1.Remove the parts of acl_3 directly, I think if we make it divided >>> > > and hierarchical, this kind of operations could be done directly in >>> > backend. >>> > > >>> > > 2.Remove acl_3, and add new acl {"group:kafka-devs"} is allowed to >>> > > READ and WRITE from {host1, host2} and {"user:alice" } is allowed >>>to >>> > > READ from {host2} >>> > > >>> > > 3.Add two denied acls,{ user:alice} is denied to WRITE from >>> > > {host1,host2} and { user:alice} is denied to READ from {host1} >>> > > >>> > > >>> > > >>> > > All these can achieve this kind of operations, but I think 1 could >>> > > more directly for user operations. If you think this optimization >>>is >>> > > not urgent, I'm also agreed. >>> > > >>> > > >>> > > >>> > > Regards >>> > > >>> > > Dapeng >>> > > >>> > > >>> > > >>> > > -----Original Message----- >>> > > >>> > > From: Parth Brahmbhatt [mailto:pbrahmbh...@hortonworks.com] >>> > > >>> > > Sent: Tuesday, April 28, 2015 12:18 AM >>> > > >>> > > To: dev@kafka.apache.org<mailto:dev@kafka.apache.org> >>> > > >>> > > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security >>> > > >>> > > >>> > > >>> > > Hi Sun, thanks for the comments, my answers are below: >>> > > >>> > > >>> > > >>> > > * I think the wiki already describes the precedence order as Deny >>> > > taking precedence over allow when conflicting acls are found >>> > > >>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizati >>> > > on+In >>> > > >>> > > terface#KIP-11-AuthorizationInterface-PermissionType >>> > > >>> > > * In the first version that I am currently writing there is no >>>group >>> > > support. Even when we add it I don't see the need to add a >>>precedence >>> > > for evaluation. it does not matter which principal matches as long >>>as >>> > > we have a match. >>> > > >>> > > * Acl storage is indexed by resource right now because that is the >>> > > primary lookup id for all authorize operations. Given acls are >>>cached >>> > > I don't see the need to optimized the storage layer any further for >>> > lookup. >>> > > >>> > > * The reason why we have acl with multi everything is to reduce >>> > > redundancy in acl storage. I am not sure how will we be able to >>>reduce >>> > > redundancy if we divide it by using one principal,one host, one >>> > operation. >>> > > >>> > > >>> > > >>> > > Thanks >>> > > >>> > > Parth >>> > > >>> > > >>> > > >>> > > On 4/26/15, 8:06 PM, "Sun, Dapeng" <dapeng....@intel.com<mailto: >>> > > dapeng....@intel.com>> wrote: >>> > > >>> > > >>> > > >>> > > >Hi Parth >>> > > >>> > > > >>> > > >>> > > >The design looks good, a few minor comments below. Since I just >>> > > >started >>> > > >>> > > >looking into the discussion and many previous discussions I may >>> > > >missed, >>> > > >>> > > >I'm sorry if these comments had be discussed. >>> > > >>> > > > >>> > > >>> > > >1. About SimpleAclAuthorizer (SimpleAuthorizer): >>> > > >>> > > >a. As my understanding, I think there should only one type >>> > > >>> > > >privilege(allow/deny) of a topic on a principle, or we make it >>>deny > >>> > > >>> > > >allow. >>> > > >>> > > >For example, acl_1 " host1 -> group1-> user1 -> read->allow" and >>> acl_2 " >>> > > >>> > > >host1-> group1 -> user1 ->read->deny", if the two acls are for a >>>same >>> > > >>> > > >topic, it may be hard to understand, do you think it's necessary >>>to >>> > > >add >>> > > >>> > > >some details about this to wiki. >>> > > >>> > > >b. And when we do authorize a user on a topic, we may should check >>> > > >>> > > >user's user level acl first, then check user's group level acl, >>> > > >finally >>> > > >>> > > >we check the host level and default level acl. do you think it's >>> > > >>> > > >necessary we add some contents like these to wiki. >>> > > >>> > > >For example, "host1 -> group1-> user1" > "host1 -> group1" > >>> "host1" >>> > > >>> > > > >>> > > >>> > > >2.About SimpleAclAuthorizer (Acl Json will be stored in zookeeper) >>>a. >>> > > >>> > > >It may be better to make acl json stored hierarchily. It may be >>>easy >>> > > >to >>> > > >>> > > >search and do authorize. For example, when we authorize a user, we >>> > > >only >>> > > >>> > > >need user related acls. >>> > > >>> > > >b. I found one acl may contains multi-principles, multi-operations >>> > > >and >>> > > >>> > > >multi-hosts, I'm strongly agreed with we provide api like these, >>>but >>> > > >>> > > >the acls stored in zookeeper or memory we may better to separate >>>to >>> > > >>> > > >one-principle, one-operation and one host. So we could make sure >>> > > >there >>> > > >>> > > >are not many acls with same meaning and make acl management >>>easily. >>> > > >>> > > > >>> > > >>> > > > >>> > > >>> > > >Regards >>> > > >>> > > >Dapeng >>> > > >>> > > > >>> > > >>> > > >-----Original Message----- >>> > > >>> > > >From: Jun Rao [mailto:j...@confluent.io] >>> > > >>> > > >Sent: Monday, April 27, 2015 5:02 AM >>> > > >>> > > >To: dev@kafka.apache.org<mailto:dev@kafka.apache.org> >>> > > >>> > > >Subject: Re: [VOTE] KIP-11- Authorization design for kafka >>>security >>> > > >>> > > > >>> > > >>> > > >A few more minor comments. >>> > > >>> > > > >>> > > >>> > > >100. To make it clear, perhaps we should rename the resource >>>"group" >>> > > >to >>> > > >>> > > >consumer-group. We can probably make the same change in CLI as >>>well >>> > > >so >>> > > >>> > > >that it's not confused with user group. >>> > > >>> > > > >>> > > >>> > > >101. Currently, create is only at the cluster level. Should it >>>also >>> > > >be >>> > > >>> > > >at topic level? For example, perhaps it's useful to allow only >>>user X >>> > > >>> > > >to create topic X. >>> > > >>> > > > >>> > > >>> > > >Thanks, >>> > > >>> > > > >>> > > >>> > > >Jun >>> > > >>> > > > >>> > > >>> > > > >>> > > >>> > > >On Sun, Apr 26, 2015 at 12:36 AM, Gwen Shapira >>><gshap...@cloudera.com >>> > > <mailto:gshap...@cloudera.com>> >>> > > >>> > > >wrote: >>> > > >>> > > > >>> > > >>> > > >> Thanks for clarifying, Parth. I think you are taking the right >>> > > >>> > > >> approach here. >>> > > >>> > > >> >>> > > >>> > > >> On Fri, Apr 24, 2015 at 11:46 AM, Parth Brahmbhatt >>> > > >>> > > >> >>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>> > > wrote: >>> > > >>> > > >> > Sorry Gwen, completely misunderstood the question :-). >>> > > >>> > > >> > >>> > > >>> > > >> > * Does everyone have the privilege to create a new Group and >>>use >>> > > >> > it >>> > > >>> > > >> > to consume from Topics he's already privileged on? >>> > > >>> > > >> > Yes in current proposal. I did not see an API to >>>create >>> > > >>> > > >> > group >>> > > >>> > > >> but if you >>> > > >>> > > >> > have a READ permission on a TOPIC and WRITE permission on that >>> > > >>> > > >> > Group you are free to join and consume. >>> > > >>> > > >> > >>> > > >>> > > >> > >>> > > >>> > > >> > * Will the CLI tool be used to manage group membership too? >>> > > >>> > > >> > Yes and I think that means I need to add ―group. >>>Updating >>> > > >>> > > >> > the >>> > > >>> > > >> KIP. Thanks >>> > > >>> > > >> > for pointing this out. >>> > > >>> > > >> > >>> > > >>> > > >> > * Groups are kind of ephemeral, right? If all consumers in the >>> > > >>> > > >> > group disconnect the group is gone, AFAIK. Do we preserve the >>> ACLs? >>> > > >>> > > >> > Or do we treat the new group as completely new resource? Can >>>we >>> > > >>> > > >> > create ACLs before the group exists, in anticipation of it >>> > > >> > getting >>> > > created? >>> > > >>> > > >> > I have considered any auto delete and auto create as >>>out >>> > > >> > of >>> > > >>> > > >> scope for the >>> > > >>> > > >> > first release. So Right now I was going with preserving the >>>acls. >>> > > >>> > > >> > Do you see any issues with this? Auto deleting would mean >>> > > >>> > > >> > authorizer will now have to get into implementation details of >>> > > >>> > > >> > kafka which I was trying to avoid. >>> > > >>> > > >> > >>> > > >>> > > >> > Thanks >>> > > >>> > > >> > Parth >>> > > >>> > > >> > >>> > > >>> > > >> > On 4/24/15, 11:33 AM, "Gwen Shapira" <gshap...@cloudera.com >>> <mailto: >>> > > gshap...@cloudera.com>> wrote: >>> > > >>> > > >> > >>> > > >>> > > >> >>We are not talking about same Groups :) >>> > > >>> > > >> >> >>> > > >>> > > >> >>I meant, Groups of consumers (which KIP-11 lists as a separate >>> > > >>> > > >> >>resource in the Privilege table) >>> > > >>> > > >> >> >>> > > >>> > > >> >>On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt >>> > > >>> > > >> >>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>> > > wrote: >>> > > >>> > > >> >>> I see Groups as something we can add incrementally in the >>> > > >> >>> current >>> > > >>> > > >> model. >>> > > >>> > > >> >>> The acls take principalType: name so groups can be >>>represented >>> > > >> >>> as >>> > > >>> > > >> group: >>> > > >>> > > >> >>> groupName. We are not managing group memberships anywhere in >>> > > >>> > > >> >>> kafka and >>> > > >>> > > >> I >>> > > >>> > > >> >>> don't see the need to do so. >>> > > >>> > > >> >>> >>> > > >>> > > >> >>> So for a topic1 using the CLI an admin can add an acl to >>>grant >>> > > >>> > > >> >>> access >>> > > >>> > > >> to >>> > > >>> > > >> >>> group:kafka-test-users. >>> > > >>> > > >> >>> >>> > > >>> > > >> >>> The authorizer implementation can have a plugin to map >>> > > >>> > > >> >>>authenticated user to groups ( This is how hadoop and storm >>> > > >>> > > >> >>>works). The plugin could be mapping user to >>>linux/ldap/active >>> > > >>> > > >> >>>directory groups but that is again upto the implementation. >>> > > >>> > > >> >>> >>> > > >>> > > >> >>> What we are offering is an interface that is extensible so >>> > > >> >>> these >>> > > >>> > > >> >>>features can be added incrementally. I can add support for >>>this >>> > > >>> > > >> >>>in the first release but don't necessarily see why this >>>would >>> > > >> >>>be >>> > > >>> > > >> >>>absolute necessity. >>> > > >>> > > >> >>> >>> > > >>> > > >> >>> Thanks >>> > > >>> > > >> >>> Parth >>> > > >>> > > >> >>> >>> > > >>> > > >> >>> On 4/24/15, 11:00 AM, "Gwen Shapira" <gshap...@cloudera.com >>> > <mailto: >>> > > gshap...@cloudera.com>> wrote: >>> > > >>> > > >> >>> >>> > > >>> > > >> >>>>Thanks. >>> > > >>> > > >> >>>> >>> > > >>> > > >> >>>>One more thing I'm missing in the KIP is details on the >>>Group >>> > > >>> > > >> >>>>resource (I think we discussed this and it was just not >>>fully >>> > > >>> > > >>updated): >>> > > >>> > > >> >>>> >>> > > >>> > > >> >>>>* Does everyone have the privilege to create a new Group and >>> > > >> >>>>use >>> > > >>> > > >> >>>>it to consume from Topics he's already privileged on? >>> > > >>> > > >> >>>>* Will the CLI tool be used to manage group membership too? >>> > > >>> > > >> >>>>* Groups are kind of ephemeral, right? If all consumers in >>>the >>> > > >>> > > >> >>>>group disconnect the group is gone, AFAIK. Do we preserve >>>the >>> > > >>> > > >> >>>>ACLs? Or do we treat the new group as completely new >>>resource? >>> > > >>> > > >> >>>>Can we create ACLs before the group exists, in anticipation >>>of >>> > > >> >>>>it >>> > > >>> > > >>getting created? >>> > > >>> > > >> >>>> >>> > > >>> > > >> >>>>Its all small details, but it will be difficult to implement >>> > > >>> > > >> >>>>KIP-11 without knowing the answers :) >>> > > >>> > > >> >>>> >>> > > >>> > > >> >>>>Gwen >>> > > >>> > > >> >>>> >>> > > >>> > > >> >>>> >>> > > >>> > > >> >>>>On Fri, Apr 24, 2015 at 9:58 AM, Parth Brahmbhatt >>> > > >>> > > >> >>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com >>> > > >> >>>>>> >>> > > wrote: >>> > > >>> > > >> >>>>> You are right, moved it to the default implementation >>>section. >>> > > >>> > > >> >>>>> >>> > > >>> > > >> >>>>> Thanks >>> > > >>> > > >> >>>>> Parth >>> > > >>> > > >> >>>>> >>> > > >>> > > >> >>>>> On 4/24/15, 9:52 AM, "Gwen Shapira" <gshap...@cloudera.com >>> > > <mailto:gshap...@cloudera.com>> wrote: >>> > > >>> > > >> >>>>> >>> > > >>> > > >> >>>>>>Sample ACL JSON and Zookeeper is in public API, but I >>>thought >>> > > >>> > > >> >>>>>>it is part of DefaultAuthorizer (Since Sentry and Argus >>>won't >>> > > >>> > > >> >>>>>>be using Zookeeper). >>> > > >>> > > >> >>>>>>Am I wrong? Or is it the KIP? >>> > > >>> > > >> >>>>>> >>> > > >>> > > >> >>>>>>On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt >>> > > >>> > > >> >>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbhatt@hortonworks.c >>> > > >> >>>>>>om>> >>> > > wrote: >>> > > >>> > > >> >>>>>>> Thanks for clarifying Gwen, KIP updated. >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>> I tried to make the distinction by creating a section >>>for >>> > > >> >>>>>>> all >>> > > >>> > > >> public >>> > > >>> > > >> >>>>>>>APIs >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriz >>> > > >> at >>> > > >>> > > >> >>>>>>>io >>> > > >>> > > >> >>>>>>>n+ >>> > > >>> > > >> >>>>>>>In >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>>>>>terface#KIP-11-AuthorizationInterface-PublicInterfacesandcla >>> > > >> >>>>>>>ss >>> > > >>> > > >> >>>>>>>e >>> > > >>> > > >> >>>>>>>s >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>> Let me know if you think there is a better way to >>>reflect >>> > this. >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>> Thanks >>> > > >>> > > >> >>>>>>> Parth >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>> On 4/24/15, 9:37 AM, "Gwen Shapira" >>><gshap...@cloudera.com >>> > > <mailto:gshap...@cloudera.com>> >>> > > >>> > > >>wrote: >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>>>>>+1 (non-binding) >>> > > >>> > > >> >>>>>>>> >>> > > >>> > > >> >>>>>>>>Two nitpicks for the wiki: >>> > > >>> > > >> >>>>>>>>* Heartbeat is probably a READ and not CLUSTER >>>operation. >>> > > >> >>>>>>>>I'm >>> > > >>> > > >> pretty >>> > > >>> > > >> >>>>>>>>sure new consumers need it to be part of a consumer >>>group. >>> > > >>> > > >> >>>>>>>>* Can you clearly separate which parts are the API >>>(common >>> > > >> >>>>>>>>to >>> > > >>> > > >> >>>>>>>>every >>> > > >>> > > >> >>>>>>>>Authorizer) and which parts are DefaultAuthorizer >>> > > >>> > > >>implementation? >>> > > >>> > > >> It >>> > > >>> > > >> >>>>>>>>will make reviews and Authorizer implementations a bit >>> > > >> >>>>>>>>easier >>> > > >>> > > >> >>>>>>>>to know exactly which is which. >>> > > >>> > > >> >>>>>>>> >>> > > >>> > > >> >>>>>>>>Gwen >>> > > >>> > > >> >>>>>>>> >>> > > >>> > > >> >>>>>>>>On Fri, Apr 24, 2015 at 9:28 AM, Parth Brahmbhatt >>> > > >>> > > >> >>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbhatt@hortonworks >>> > > >> >>>>>>>>.com>> >>> > > wrote: >>> > > >>> > > >> >>>>>>>>> Hi, >>> > > >>> > > >> >>>>>>>>> >>> > > >>> > > >> >>>>>>>>> I would like to open KIP-11 for voting. >>> > > >>> > > >> >>>>>>>>> >>> > > >>> > > >> >>>>>>>>> Thanks >>> > > >>> > > >> >>>>>>>>> Parth >>> > > >>> > > >> >>>>>>>>> >>> > > >>> > > >> >>>>>>>>> On 4/22/15, 1:56 PM, "Parth Brahmbhatt" >>> > > >>> > > >> >>>>>>>>><pbrahmbh...@hortonworks.com<mailto: >>> > > pbrahmbh...@hortonworks.com>> >>> > > >>> > > >> >>>>>>>>> wrote: >>> > > >>> > > >> >>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>Hi Jeff, >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>Thanks a lot for the review. I think you have a valid >>> > > >> >>>>>>>>>>point >>> > > >>> > > >> >>>>>>>>>>about acls being duplicated and the simplest solution >>> > > >> >>>>>>>>>>would >>> > > >>> > > >> >>>>>>>>>>be to modify >>> > > >>> > > >> acls >>> > > >>> > > >> >>>>>>>>>>class >>> > > >>> > > >> >>>>>>>>>>so they hold a set of principals instead of single >>> > > >>> > > >> >>>>>>>>>>principal. i.e >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>><user_a,user_b> has <READ,WRITE,DESCRIBE> Permissions >>>on >>> > > >>> > > >> >>>>>>>>>><Topic1> from <Host1, Host2, Host3>. >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>I think the evaluation order only matters for the >>> > > >>> > > >> >>>>>>>>>>permissionType which is Deny acls should be evaluated >>> > > >>> > > >> >>>>>>>>>>before allow acls. To give you an example suppose we >>>have >>> > > >>> > > >> >>>>>>>>>>following acls >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>acl1 -> user1 is allowed to READ from all hosts. >>> > > >>> > > >> >>>>>>>>>>acl2 -> host1 is allowed to READ regardless of who is >>>the >>> > > >>> > > >>user. >>> > > >>> > > >> >>>>>>>>>>acl3 -> host2 is allowed to READ regardless of who is >>>the >>> > > >>> > > >>user. >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>acl4 -> user1 is denied to READ from host1. >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>As stated in the KIP we first evaluate DENY so if >>>user1 >>> > > >>> > > >> >>>>>>>>>>tries to access from host1 he will be denied(acl4), >>>even >>> > > >>> > > >> >>>>>>>>>>though both user1 and >>> > > >>> > > >> >>>>>>>>>>host1 >>> > > >>> > > >> >>>>>>>>>>has >>> > > >>> > > >> >>>>>>>>>>acl's for allow with wildcards (acl1, acl2). >>> > > >>> > > >> >>>>>>>>>>If user1 tried to READ from host2 , the action will be >>> > > >>> > > >> >>>>>>>>>>allowed >>> > > >>> > > >> and >>> > > >>> > > >> >>>>>>>>>>it >>> > > >>> > > >> >>>>>>>>>>does >>> > > >>> > > >> >>>>>>>>>>not matter if we match acl3 or acl1 so I don't think >>>the >>> > > >>> > > >> >>>>>>>>>>evaluation order matters here. >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>"Will people actually use hosts with users?" I really >>> > > >> >>>>>>>>>>don't >>> > > >>> > > >> >>>>>>>>>>know but given ACl's are part of our Public APIs I >>> > > >> >>>>>>>>>>thought >>> > > >>> > > >> >>>>>>>>>>it is better to try and cover more use cases. If >>>others >>> > > >>> > > >> >>>>>>>>>>think this extra complexity is not >>> > > >>> > > >> worth >>> > > >>> > > >> >>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>value its adding please raise your concerns so we can >>> > > >>> > > >> >>>>>>>>>>discuss if it should be removed from the acl >>>structure. >>> > > >>> > > >> >>>>>>>>>>Note that even in absence of hosts from ACL users will >>> > > >>> > > >> >>>>>>>>>>still be able to whitelist/blacklist host as long as >>>we >>> > > >>> > > >> >>>>>>>>>>start supporting principalType = "host", easy to add >>>and >>> > > >>> > > >> >>>>>>>>>>can be >>> > > >>> > > >> an >>> > > >>> > > >> >>>>>>>>>>incremental improvement. They will however loose the >>> > > >>> > > >> >>>>>>>>>>ability to restrict access to users just from a set of >>> > hosts. >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>We agreed to offer a CLI to overcome the JSON acl >>>config >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authori >>> > > >>> > > >> >>>>>>>>>>za >>> > > >>> > > >> >>>>>>>>>>ti >>> > > >>> > > >> >>>>>>>>>>on >>> > > >>> > > >> >>>>>>>>>>+I >>> > > >>> > > >> >>>>>>>>>>n >>> > > >>> > > >> >>>>>>>>>>>>>terface#KIP-11-AuthorizationInterface-AclManagement(CLI). >>> > > >> >>>>>>>>>>I >>> > > >>> > > >> >>>>>>>>>>still like Jsons but that probably has something to do >>> > > >> >>>>>>>>>>with >>> > > >>> > > >> >>>>>>>>>>me being a developer :-). >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>Thanks >>> > > >>> > > >> >>>>>>>>>>Parth >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>On 4/22/15, 11:38 AM, "Jeff Holoman" >>> > > >>> > > >> >>>>>>>>>><jholo...@cloudera.com<mailto:jholo...@cloudera.com>> >>> > > >>> > > >> >>>>>>>>>>wrote: >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>Parth, >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>This is a long thread, so trying to keep up here, >>>sorry >>> > > >> >>>>>>>>>>>if >>> > > >>> > > >> >>>>>>>>>>>this has been covered before. First, great job on the >>> > > >> >>>>>>>>>>>KIP >>> > > >>> > > >> >>>>>>>>>>>proposal and work so far. >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>Are we sure that we want to tie host level access to >>>a >>> > > >>> > > >> >>>>>>>>>>>given user? >>> > > >>> > > >> >>>>>>>>>>>My >>> > > >>> > > >> >>>>>>>>>>>understanding is that the ACL will be (omitting some >>> > > >>> > > >> >>>>>>>>>>>fields) >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>user_a, host1, host2, host3 user_b, host1, host2, >>>host3 >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>So there would potentially be a lot of redundancy in >>>the >>> > > >>> > > >> configs. >>> > > >>> > > >> >>>>>>>>>>>Does >>> > > >>> > > >> >>>>>>>>>>>it >>> > > >>> > > >> >>>>>>>>>>>make sense to have hosts be at the same level as >>> > > >> >>>>>>>>>>>principal >>> > > >>> > > >> >>>>>>>>>>>in >>> > > >>> > > >> the >>> > > >>> > > >> >>>>>>>>>>>hierarchy? This way you could just blanket the >>>allowed / >>> > > >>> > > >> >>>>>>>>>>>denied hosts and only have to worry about the users. >>>So >>> > > >> >>>>>>>>>>>if >>> > > >>> > > >> >>>>>>>>>>>you follow this, then >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>we can wildcard the user so we can have a separate >>>list >>> > > >> >>>>>>>>>>>of >>> > > >>> > > >> >>>>>>>>>>>just host-based access. What's the order that the >>>perms >>> > > >>> > > >> >>>>>>>>>>>would be evaluated if a there was more than one match >>>on >>> > > >> >>>>>>>>>>>a >>> > > >>> > > >> >>>>>>>>>>>principal ? >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>Is the thought that there wouldn't usually be much >>> > > >> >>>>>>>>>>>overlap >>> > > >>> > > >> >>>>>>>>>>>on hosts? >>> > > >>> > > >> >>>>>>>>>>>I >>> > > >>> > > >> >>>>>>>>>>>guess I can imagine a scenario where I want to >>> > > >>> > > >> >>>>>>>>>>>offline/online access to a particular hosts or set of >>> > > >>> > > >> >>>>>>>>>>>hosts and if there was overlap, I'm doing a bunch of >>> > > >> >>>>>>>>>>>alter >>> > > >>> > > >> >>>>>>>>>>>commands for just a single host. Maybe this is >>> > > >>> > > >> too >>> > > >>> > > >> >>>>>>>>>>>contrived >>> > > >>> > > >> >>>>>>>>>>>an example? >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>I agree that having this level of granularity gives >>> > > >>> > > >> >>>>>>>>>>>flexibility but I wonder if people will actually use >>>it >>> > > >>> > > >> >>>>>>>>>>>and not just * the hosts for a given user and create >>> > > >>> > > >> >>>>>>>>>>>separate "global" list as i mentioned above? >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>The only other system I know of that ties users with >>> > > >> >>>>>>>>>>>hosts >>> > > >>> > > >> >>>>>>>>>>>for access is MySql and I don't love that model. >>> > > >> >>>>>>>>>>>Companies >>> > > >>> > > >> >>>>>>>>>>>usually standardize on group authorization anyway, >>>are >>> > > >> >>>>>>>>>>>we >>> > > >>> > > >> >>>>>>>>>>>complicating that issue with the inclusion of hosts >>> > > >>> > > >> >>>>>>>>>>>attached to users? Additionally I worry about the >>>debt >>> > > >> >>>>>>>>>>>of >>> > > >>> > > >> >>>>>>>>>>>big JSON configs in the first place, most >>>non-developers >>> > > >>> > > >> >>>>>>>>>>>find them non-intuitive already, so anything to ease >>> > > >> >>>>>>>>>>>this >>> > > >>> > > >> >>>>>>>>>>>I think would be beneficial. >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>Thanks >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>Jeff >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>On Wed, Apr 22, 2015 at 2:22 PM, Parth Brahmbhatt < >>> > > >>> > > >> >>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto: >>> > > pbrahmbh...@hortonworks.com>> wrote: >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> Sorry I missed your last questions. I am +0 on >>>adding >>> > > >>> > > >> >>>>>>>>>>>>―host option for ―list, we could add it for >>>symmetry. >>> > > >>> > > >> >>>>>>>>>>>>Again if this is only a CLI change it can be added >>> > > >> >>>>>>>>>>>>later >>> > > >>> > > >> >>>>>>>>>>>>if you mean adding this in authorizer interface then >>>we >>> > > >>> > > >> >>>>>>>>>>>>should make a decision now. >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> Given a choice I would like to actually keep only >>>one >>> > > >>> > > >> >>>>>>>>>>>>option which is resource based get (remove even the >>> > > >> >>>>>>>>>>>>get >>> > > >>> > > >> >>>>>>>>>>>>based on principal). I see those (getAcl for >>>principal >>> > > >>> > > >> >>>>>>>>>>>>or >>> > > >>> > > >> >>>>>>>>>>>>host) as special filtering case which can easily be >>> > > >>> > > >> >>>>>>>>>>>>achieved by a third party tool by doing "list all >>> topics" >>> > > >>> > > >> >>>>>>>>>>>>and >>> > > >>> > > >> >>>>>>>>>>>>calling >>> > > >>> > > >> >>>>>>>>>>>> getAcls for each topic and applying filtering logic >>>on >>> > > >>> > > >>that. >>> > > >>> > > >> I >>> > > >>> > > >> >>>>>>>>>>>>really >>> > > >>> > > >> >>>>>>>>>>>> don't see the need to make those first class >>>citizens >>> > > >> >>>>>>>>>>>> of >>> > > >>> > > >> >>>>>>>>>>>>the authorizer interface given these kind of >>>queries >>> > > >>> > > >> >>>>>>>>>>>>will be issued outside >>> > > >>> > > >> of >>> > > >>> > > >> >>>>>>>>>>>>broker >>> > > >>> > > >> >>>>>>>>>>>>JVM >>> > > >>> > > >> >>>>>>>>>>>> so they will not benefit from the caching and >>>because >>> > > >>> > > >> >>>>>>>>>>>>the storage will be indexed on resource both these >>> > > >>> > > >> >>>>>>>>>>>>options even as a first class API will just scan >>>all >>> > > >>> > > >> >>>>>>>>>>>>topic acls and apply filtering logic. >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> Thanks >>> > > >>> > > >> >>>>>>>>>>>> Parth >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> On 4/22/15, 11:08 AM, "Parth Brahmbhatt" >>> > > >>> > > >> >>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto: >>> > > pbrahmbh...@hortonworks.com>> >>> > > >>> > > >> >>>>>>>>>>>> wrote: >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> >Please see all the available options here >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Autho >>> > > >>> > > >> >>>>>>>>>>>>ri >>> > > >>> > > >> >>>>>>>>>>>>za >>> > > >>> > > >> >>>>>>>>>>>>ti >>> > > >>> > > >> >>>>>>>>>>>>on >>> > > >>> > > >> >>>>>>>>>>>>+ >>> > > >>> > > >> >>>>>>>>>>>>I >>> > > >>> > > >> >>>>>>>>>>>> >>>>nterface#KIP-11-AuthorizationInterface-AclManagement( >>> > > >> >>>>>>>>>>>> >CL >>> > > >>> > > >> >>>>>>>>>>>> >I >>> > > >>> > > >> >>>>>>>>>>>> >) . I >>> > > >>> > > >> >>>>>>>>>>>>think >>> > > >>> > > >> >>>>>>>>>>>>it >>> > > >>> > > >> >>>>>>>>>>>> >covers both hosts and operations and allows to >>> > > >> >>>>>>>>>>>> >specify >>> > > >>> > > >> >>>>>>>>>>>> >a list >>> > > >>> > > >> >>>>>>>>>>>>for >>> > > >>> > > >> >>>>>>>>>>>>both. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >Thanks >>> > > >>> > > >> >>>>>>>>>>>> >Parth >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >From: Tom Graves >>> > > >>> > > >> >>>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com >>> <mailto: >>> > > tgraves...@yahoo.com%3cmailto:tgraves...@yahoo.com>>> >>> > > >>> > > >> >>>>>>>>>>>> >Reply-To: Tom Graves >>> > > >>> > > >> >>>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com >>> <mailto: >>> > > tgraves...@yahoo.com%3cmailto:tgraves...@yahoo.com>>> >>> > > >>> > > >> >>>>>>>>>>>> >Date: Wednesday, April 22, 2015 at 11:02 AM >>> > > >>> > > >> >>>>>>>>>>>> >To: Parth Brahmbhatt >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto: >>> > > >>> > > >> pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com> >>> > > >>> > > >> >>>>>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>>, >>> > > >>> > > >> >>>>>>>>>>>> >"dev@kafka.apache.org<mailto:dev@kafka.apache.org >>> > > ><mailto:dev@kafka.apache.org%3cmailto:dev@kafka.apache.org%3e>" >>> > > >>> > > >> >>>>>>>>>>>> ><dev@kafka.apache.org<mailto:dev@kafka.apache.org >>> > <mailto: >>> > > dev@kafka.apache.org%3cmailto:dev@kafka.apache.org>>> >>> > > >>> > > >> >>>>>>>>>>>> >Subject: Re: [DISCUSS] KIP-11- Authorization >>>design >>> > > >> >>>>>>>>>>>> >for >>> > > >>> > > >> >>>>>>>>>>>> >kafka >>> > > >>> > > >> >>>>>>>>>>>>security >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >Thanks for the explanations Parth. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >On the configs questions, the way I see it is its >>> > > >> >>>>>>>>>>>> >more >>> > > >>> > > >> >>>>>>>>>>>> >likely >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >accidentally give everyone access, especially >>>since >>> > > >> >>>>>>>>>>>> >you >>> > > >>> > > >> >>>>>>>>>>>> >have >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>>run >>> > > >>> > > >> >>>>>>>>>>>>a >>> > > >>> > > >> >>>>>>>>>>>> >separate command to change the acls. If there was >>> > > >> >>>>>>>>>>>> >some >>> > > >>> > > >> >>>>>>>>>>>> >config >>> > > >>> > > >> >>>>>>>>>>>>for >>> > > >>> > > >> >>>>>>>>>>>> >defaults, a cluster admin could change that to be >>> > > >>> > > >> >>>>>>>>>>>> >nobody or >>> > > >>> > > >> >>>>>>>>>>>>certain >>> > > >>> > > >> >>>>>>>>>>>>set >>> > > >>> > > >> >>>>>>>>>>>> >of users, then grant others permissions. This >>>would >>> > > >>> > > >> >>>>>>>>>>>> >also >>> > > >>> > > >> >>>>>>>>>>>>remove >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>>race >>> > > >>> > > >> >>>>>>>>>>>> >between commands. This is something you can >>>always >>> > > >> >>>>>>>>>>>> >add >>> > > >>> > > >> >>>>>>>>>>>> >later >>> > > >>> > > >> >>>>>>>>>>>>though >>> > > >>> > > >> >>>>>>>>>>>>if >>> > > >>> > > >> >>>>>>>>>>>> >people request it. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >So in kafka-acl.sh how do I actually tell it what >>>the >>> > > >>> > > >> >>>>>>>>>>>>operation >>> > > >>> > > >> >>>>>>>>>>>>is? >>> > > >>> > > >> >>>>>>>>>>>> >kafka-acl.sh --topic testtopic --add >>>--grandprincipal >>> > > >>> > > >> >>>>>>>>>>>>user:joe,user:kate >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >where does READ, WRITE, etc go? Can specify as a >>> > > >> >>>>>>>>>>>> >list >>> > > >>> > > >> >>>>>>>>>>>> >so I >>> > > >>> > > >> >>>>>>>>>>>>don't >>> > > >>> > > >> >>>>>>>>>>>>have >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >run this a bunch of times for each. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >Do you want to have a --host option for --list so >>> > > >> >>>>>>>>>>>> >that >>> > > >>> > > >> >>>>>>>>>>>> >admins >>> > > >>> > > >> >>>>>>>>>>>>could >>> > > >>> > > >> >>>>>>>>>>>>see >>> > > >>> > > >> >>>>>>>>>>>> >what acls apply to specific host(s)? >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >Tom >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >On Wednesday, April 22, 2015 11:38 AM, Parth >>> > > >> >>>>>>>>>>>> >Brahmbhatt >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto: >>> > > >>> > > >> pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com> >>> > > >>> > > >> >>>>>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>wrote: >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >FYI, I have modified the KIP to include group as >>> > > >>> > > >> >>>>>>>>>>>> >resource. In >>> > > >>> > > >> >>>>>>>>>>>>order >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >access "joinGroup" and "commitOFfset" APIs the >>>user >>> > > >>> > > >> >>>>>>>>>>>> >will need >>> > > >>> > > >> >>>>>>>>>>>>a >>> > > >>> > > >> >>>>>>>>>>>>read >>> > > >>> > > >> >>>>>>>>>>>> >permission on topic and WRITE permission on group. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >I plan to open a VOTE thread by noon if there are >>>no >>> > > >>> > > >> >>>>>>>>>>>> >more >>> > > >>> > > >> >>>>>>>>>>>>concerns. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >Thanks >>> > > >>> > > >> >>>>>>>>>>>> >Parth >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >On 4/22/15, 9:03 AM, "Tom Graves" >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>><tgraves...@yahoo.com.INVALID<mailto: >>> > > >>> > > >> tgraves...@yahoo.com.INVAL<mailto:tgraves...@yahoo.com.INVAL> >>> > > >>> > > >> >>>>>>>>>>>>>ID >>> > > >>> > > >> >>>>>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> wrote: >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>Hey everyone, >>> > > >>> > > >> >>>>>>>>>>>> >>Sorry to jump in on the conversation so late. I'm >>> > > >> >>>>>>>>>>>> >>new >>> > > >>> > > >> >>>>>>>>>>>> >>to >>> > > >>> > > >> >>>>>>>>>>>>Kafka. >>> > > >>> > > >> >>>>>>>>>>>>I'll >>> > > >>> > > >> >>>>>>>>>>>> >>apologize in advance if you have already covered >>> > > >> >>>>>>>>>>>> >>some >>> > > >>> > > >> >>>>>>>>>>>> >>of my >>> > > >>> > > >> >>>>>>>>>>>>questions. I >>> > > >>> > > >> >>>>>>>>>>>> >>read through the wiki and had some comments and >>> > > >>> > > >>questions. >>> > > >>> > > >> >>>>>>>>>>>> >>1) public enum Operation needs EDIT changed to >>>ALTER >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >> Done. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>2) Does the Authorizer class need a setAcls? >>>Rather >>> > > >>> > > >> >>>>>>>>>>>> >>then >>> > > >>> > > >> >>>>>>>>>>>>just >>> > > >>> > > >> >>>>>>>>>>>>add >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>>be >>> > > >>> > > >> >>>>>>>>>>>> >>able to set to explicit list and overwrite what >>>was >>> > > >>> > > >>there? >>> > > >>> > > >> I >>> > > >>> > > >> >>>>>>>>>>>>see >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>> >>kafka-acl.sh lists a removeall so I guess you >>>could >>> > > >> >>>>>>>>>>>> >>do >>> > > >>> > > >> >>>>>>>>>>>>removeall >>> > > >>> > > >> >>>>>>>>>>>>and >>> > > >>> > > >> >>>>>>>>>>>>then >>> > > >>> > > >> >>>>>>>>>>>> >>add. I also don't see a removeall in the >>>Authorizer >>> > > >>> > > >> >>>>>>>>>>>> >>class, >>> > > >>> > > >> >>>>>>>>>>>>is >>> > > >>> > > >> >>>>>>>>>>>>it >>> > > >>> > > >> >>>>>>>>>>>>going >>> > > >>> > > >> >>>>>>>>>>>> >>to loop through them all to remove each one? >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > There is an overloaded version of removeAcls >>>in >>> > > >> >>>>>>>>>>>> > the >>> > > >>> > > >> >>>>>>>>>>>>interface >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>> >takes >>> > > >>> > > >> >>>>>>>>>>>> >in resource as the only input and as described in >>>the >>> > > >>> > > >> >>>>>>>>>>>> >javadoc >>> > > >>> > > >> >>>>>>>>>>>>all >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>>acls >>> > > >>> > > >> >>>>>>>>>>>> >attached to that resource will be deleted. To >>>cover >>> > > >> >>>>>>>>>>>> >the >>> > > >>> > > >> setAcl >>> > > >>> > > >> >>>>>>>>>>>>use >>> > > >>> > > >> >>>>>>>>>>>>case >>> > > >>> > > >> >>>>>>>>>>>> >the caller can first call remove and then add. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>3) Can someone tell me what the use case to do >>>acls >>> > > >>> > > >> >>>>>>>>>>>> >>based on >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>>hosts? >>> > > >>> > > >> >>>>>>>>>>>> >>I can see some possibilities just wondering if we >>> > > >> >>>>>>>>>>>> >>can >>> > > >>> > > >> >>>>>>>>>>>>concrete >>> > > >>> > > >> >>>>>>>>>>>>ones >>> > > >>> > > >> >>>>>>>>>>>>where >>> > > >>> > > >> >>>>>>>>>>>> >>one user is allowed from one host but not >>>another. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > I am not sure if I understand the question >>>given >>> > > >>> > > >> >>>>>>>>>>>> > the use >>> > > >>> > > >> >>>>>>>>>>>>case >>> > > >>> > > >> >>>>>>>>>>>>you >>> > > >>> > > >> >>>>>>>>>>>> >described in your question is what we are trying >>>to >>> > > >>> > > >> >>>>>>>>>>>> >cover >>> > > >>> > > >> with >>> > > >>> > > >> >>>>>>>>>>>>use >>> > > >>> > > >> >>>>>>>>>>>>of >>> > > >>> > > >> >>>>>>>>>>>> >hosts in Acl. There are some additional use cases >>> > > >> >>>>>>>>>>>> >like >>> > > >>> > > >> >>>>>>>>>>>> >"allow >>> > > >>> > > >> >>>>>>>>>>>>access >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >any user from host1,host2" but I think primarily >>>it >>> > > >>> > > >> >>>>>>>>>>>> >gives the >>> > > >>> > > >> >>>>>>>>>>>>admins >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>> >ability to define acls at a more granular level. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>4) I'm a bit unclear how the "resource" works in >>>the >>> > > >>> > > >> >>>>>>>>>>>>Authorizer >>> > > >>> > > >> >>>>>>>>>>>>class. >>> > > >>> > > >> >>>>>>>>>>>> >>From what I see we have 2 resources - topics and >>> > cluster. >>> > > >>> > > >> >>>>>>>>>>>>If I >>> > > >>> > > >> >>>>>>>>>>>>want >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >>add an acl to allow "joe" to CREATE for the >>>cluster >>> > > >>> > > >> >>>>>>>>>>>> >>then I >>> > > >>> > > >> >>>>>>>>>>>>call >>> > > >>> > > >> >>>>>>>>>>>>addAcls >>> > > >>> > > >> >>>>>>>>>>>> >>with Acl("user: joe", ALLOW, Set(*), >>>Set(CREATE)) >>> > > >> >>>>>>>>>>>> >>and >>> > > >>> > > >> >>>>>>>>>>>>"cluster"? >>> > > >>> > > >> >>>>>>>>>>>>What >>> > > >>> > > >> >>>>>>>>>>>> >>if I want to call addAcls for DESCRIBE on a >>>topic? >>> > > >> >>>>>>>>>>>> >>Is >>> > > >>> > > >> >>>>>>>>>>>> >>the >>> > > >>> > > >> >>>>>>>>>>>>resource >>> > > >>> > > >> >>>>>>>>>>>>then >>> > > >>> > > >> >>>>>>>>>>>> >>"topic" or is it the topic name? >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > We now have 3 resources(added group), please >>>see >>> > > >>> > > >> >>>>>>>>>>>> > the >>> > > >>> > > >> >>>>>>>>>>>>updated >>> > > >>> > > >> >>>>>>>>>>>>doc. >>> > > >>> > > >> >>>>>>>>>>>>The >>> > > >>> > > >> >>>>>>>>>>>> >CREATE acl that you described is correct. For any >>> > > >> >>>>>>>>>>>> >topic >>> > > >>> > > >> >>>>>>>>>>>>operation >>> > > >>> > > >> >>>>>>>>>>>>you >>> > > >>> > > >> >>>>>>>>>>>> >should use topic name as the resource name and for >>> > > >>> > > >> >>>>>>>>>>>> >group the >>> > > >>> > > >> >>>>>>>>>>>>user >>> > > >>> > > >> >>>>>>>>>>>>will >>> > > >>> > > >> >>>>>>>>>>>> >provide groupId as resource name. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>5) reassigning partitions is a CLUSTER_ACTION or >>> > > >>> > > >>superuser? >>> > > >>> > > >> >>>>>>>>>>>>Its >>> > > >>> > > >> >>>>>>>>>>>>not >>> > > >>> > > >> >>>>>>>>>>>> >>totally clear to me the differences between >>>these. >>> > > >>> > > >> >>>>>>>>>>>> >>what >>> > > >>> > > >> >>>>>>>>>>>>about >>> > > >>> > > >> >>>>>>>>>>>>increasing >>> > > >>> > > >> >>>>>>>>>>>> >># of partitions? >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > I see this as an alter topic operation so it >>>is >>> > > >> >>>>>>>>>>>> > at >>> > > >>> > > >> >>>>>>>>>>>> > topic >>> > > >>> > > >> >>>>>>>>>>>>level >>> > > >>> > > >> >>>>>>>>>>>>and >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>> >user must have alter permissions on topic. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>6) groups are mentioned, are we supporting right >>> > > >> >>>>>>>>>>>> >>away >>> > > >>> > > >> >>>>>>>>>>>> >>or is >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>>a >>> > > >>> > > >> >>>>>>>>>>>>follow >>> > > >>> > > >> >>>>>>>>>>>> >>on item? (is there going to be a >>>kafka.supergroups) >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > I think it can be a separate jira just for >>> > > >> >>>>>>>>>>>> > braking >>> > > >>> > > >> >>>>>>>>>>>> > down >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>>code >>> > > >>> > > >> >>>>>>>>>>>> >review >>> > > >>> > > >> >>>>>>>>>>>> >in smaller chunk. We will support it in first >>>version >>> > > >>> > > >> >>>>>>>>>>>> >but I >>> > > >>> > > >> >>>>>>>>>>>>think >>> > > >>> > > >> >>>>>>>>>>>>if >>> > > >>> > > >> >>>>>>>>>>>>we >>> > > >>> > > >> >>>>>>>>>>>> >can not do it for any reason that should not >>>block >>>a >>> > > >>> > > >> >>>>>>>>>>>> >release >>> > > >>> > > >> >>>>>>>>>>>>with >>> > > >>> > > >> >>>>>>>>>>>>all >>> > > >>> > > >> >>>>>>>>>>>>the >>> > > >>> > > >> >>>>>>>>>>>> >other authZ work. We made deliberate design >>>choices >>> > > >>> > > >> >>>>>>>>>>>> >(like >>> > > >>> > > >> >>>>>>>>>>>>introducing >>> > > >>> > > >> >>>>>>>>>>>>a >>> > > >>> > > >> >>>>>>>>>>>> >principalType in KafkaPrinciapl) to allow >>>supporting >>> > > >>> > > >> >>>>>>>>>>>> >groups >>> > > >>> > > >> as >>> > > >>> > > >> >>>>>>>>>>>>an >>> > > >>> > > >> >>>>>>>>>>>> >incremental change. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>7) Are there config options for setting acls >>>when >>>I >>> > > >>> > > >> >>>>>>>>>>>> >>create >>> > > >>> > > >> my >>> > > >>> > > >> >>>>>>>>>>>>topic? >>> > > >>> > > >> >>>>>>>>>>>>Or >>> > > >>> > > >> >>>>>>>>>>>> >>do I have to create my topic and then run the >>> > > >>> > > >> >>>>>>>>>>>> >>kafka-acl.sh >>> > > >>> > > >> >>>>>>>>>>>>script >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>>set >>> > > >>> > > >> >>>>>>>>>>>> >>them? Although its very small, there would be >>> > > >>> > > >> >>>>>>>>>>>> >>possible race >>> > > >>> > > >> >>>>>>>>>>>>there >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>> >>someone could start producing to topic before >>>acls >>> > > >> >>>>>>>>>>>> >>are >>> > > >>> > > >>set. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > We discussed this yesterday and we agreed to >>>go >>> > > >>> > > >> >>>>>>>>>>>> > with >>> > > >>> > > >> >>>>>>>>>>>>kafka-acl.sh. >>> > > >>> > > >> >>>>>>>>>>>>Yes >>> > > >>> > > >> >>>>>>>>>>>> >there is a very very small window of vulnerability >>> > > >> >>>>>>>>>>>> >but >>> > > >>> > > >> >>>>>>>>>>>> >I >>> > > >>> > > >> think >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>>really >>> > > >>> > > >> >>>>>>>>>>>> >does not warrant to change the decision in this >>>case. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>8) are there configs for cluster level acl >>>defaults? >>> > > >>> > > >> >>>>>>>>>>>> >>Or >>> > > >>> > > >> does >>> > > >>> > > >> >>>>>>>>>>>>it >>> > > >>> > > >> >>>>>>>>>>>>default >>> > > >>> > > >> >>>>>>>>>>>> >>to superusers on bringing up new cluster and you >>> > > >> >>>>>>>>>>>> >>have >>> > > >>> > > >> >>>>>>>>>>>> >>to >>> > > >>> > > >> >>>>>>>>>>>>modify >>> > > >>> > > >> >>>>>>>>>>>>with >>> > > >>> > > >> >>>>>>>>>>>>cli. >>> > > >>> > > >> >>>>>>>>>>>> >>thanks,Tom >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > No defaults, the default is superusers will >>>have >>> > > >>> > > >> >>>>>>>>>>>> > full >>> > > >>> > > >> >>>>>>>>>>>>access. >>> > > >>> > > >> >>>>>>>>>>>>I >>> > > >>> > > >> >>>>>>>>>>>>don't >>> > > >>> > > >> >>>>>>>>>>>> >think making assumptions about ones security >>> > > >>> > > >> >>>>>>>>>>>> >requirement >>> > > >>> > > >> >>>>>>>>>>>>should >>> > > >>> > > >> >>>>>>>>>>>>be >>> > > >>> > > >> >>>>>>>>>>>>our >>> > > >>> > > >> >>>>>>>>>>>> >burden. >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >> On Tuesday, April 21, 2015 7:10 PM, Parth >>> > > >>> > > >> >>>>>>>>>>>> >> Brahmbhatt >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto: >>> > > >>> > > >> pbrahmbh...@hortonworks.co<mailto:pbrahmbh...@hortonworks.co> >>> > > >>> > > >> >>>>>>>>>>>>>>m> >>> > > >>> > > >> >>>>>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>>wrote: >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >> I have added the notes to KIP-11 Open question >>> > sections. >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>Thanks >>> > > >>> > > >> >>>>>>>>>>>> >>Parth >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>On 4/21/15, 4:49 PM, "Gwen Shapira" >>> > > >>> > > >> >>>>>>>>>>>> >>>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com >>> > > <mailto:gshap...@cloudera.com%3cmailto:gshap...@cloudera.com>>> >>> > > >>> > > >> wrote: >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>Adding my notes from today's call to the thread: >>> > > >>> > > >> >>>>>>>>>>>> >>> >>> > > >>> > > >> >>>>>>>>>>>> >>>** Deny or Allow all by default? We will add a >>> > > >>> > > >> configuration >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >>>control this. The configuration will default to >>> > > >>> > > >> >>>>>>>>>>>> >>>"allow" for >>> > > >>> > > >> >>>>>>>>>>>>backward >>> > > >>> > > >> >>>>>>>>>>>> >>>compatibility. Security admins can set it to >>>"deny" >>> > > >>> > > >> >>>>>>>>>>>> >>> >>> > > >>> > > >> >>>>>>>>>>>> >>>** Storing ACLs for default authorizers: We'll >>> > > >> >>>>>>>>>>>> >>>store >>> > > >>> > > >> >>>>>>>>>>>> >>>them >>> > > >>> > > >> in >>> > > >>> > > >> >>>>>>>>>>>>ZK. >>> > > >>> > > >> >>>>>>>>>>>>We'll >>> > > >>> > > >> >>>>>>>>>>>> >>>support pointing the authorizer to any ZK. >>> > > >>> > > >> >>>>>>>>>>>> >>>The use of ZK will be internal to the default >>> > > >>> > > >>authorizer. >>> > > >>> > > >> >>>>>>>>>>>>Authorizer >>> > > >>> > > >> >>>>>>>>>>>> >>>reads ACLs from cache every hour. We proposed >>> > > >> >>>>>>>>>>>> >>>having >>> > > >>> > > >> >>>>>>>>>>>>mechanism >>> > > >>> > > >> >>>>>>>>>>>> >>>(possibly via new ZK node) to tell broker to >>> > > >> >>>>>>>>>>>> >>>refresh >>> > > >>> > > >> >>>>>>>>>>>> >>>the >>> > > >>> > > >> >>>>>>>>>>>>cache >>> > > >>> > > >> >>>>>>>>>>>> >>>immediately. >>> > > >>> > > >> >>>>>>>>>>>> >>> >>> > > >>> > > >> >>>>>>>>>>>> >>>** Support deny as permission type - we agreed >>>to >>> > > >>> > > >> >>>>>>>>>>>> >>>keep >>> > > >>> > > >> this. >>> > > >>> > > >> >>>>>>>>>>>> >>> >>> > > >>> > > >> >>>>>>>>>>>> >>>** Mapping operations to API: We may need to add >>> > > >>> > > >> >>>>>>>>>>>> >>>Group as a >>> > > >>> > > >> >>>>>>>>>>>>resource, >>> > > >>> > > >> >>>>>>>>>>>> >>>with JoinGroup and OffsetCommit require >>>privilege >>> > > >> >>>>>>>>>>>> >>>on >>> > > >>> > > >> >>>>>>>>>>>> >>>the >>> > > >>> > > >> >>>>>>>>>>>>consumer >>> > > >>> > > >> >>>>>>>>>>>> >>>group. >>> > > >>> > > >> >>>>>>>>>>>> >>>This can be something we pass now and >>>authorizers >>> > > >> >>>>>>>>>>>> >>>can >>> > > >>> > > >> >>>>>>>>>>>>support >>> > > >>> > > >> >>>>>>>>>>>>in >>> > > >>> > > >> >>>>>>>>>>>> >>>future. - Jay will write specifics to the >>>mailing >>> > > >>> > > >> >>>>>>>>>>>> >>>list >>> > > >>> > > >> >>>>>>>>>>>>discussion. >>> > > >>> > > >> >>>>>>>>>>>> >>> >>> > > >>> > > >> >>>>>>>>>>>> >>>On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps >>> > > >>> > > >> >>>>>>>>>>>> >>><jay.kr...@gmail.com<mailto:jay.kr...@gmail.com >>> > <mailto: >>> > > jay.kr...@gmail.com%3cmailto:jay.kr...@gmail.com>>> wrote: >>> > > >>> > > >> >>>>>>>>>>>> >>>> Following up on the KIP discussion. Two >>>options >>> > > >> >>>>>>>>>>>> >>>> for >>> > > >>> > > >> >>>>>>>>>>>>authorizing >>> > > >>> > > >> >>>>>>>>>>>> >>>>consumers >>> > > >>> > > >> >>>>>>>>>>>> >>>> to read topic "t" as part of group "g": >>> > > >>> > > >> >>>>>>>>>>>> >>>> 1. READ permission on resource /topic/t 2. >>>READ >>> > > >>> > > >> >>>>>>>>>>>> >>>>permission on resource /topic/t AND WRITE >>> > > >>> > > >> >>>>>>>>>>>>permission >>> > > >>> > > >> >>>>>>>>>>>>on >>> > > >>> > > >> >>>>>>>>>>>> >>>>/group/g >>> > > >>> > > >> >>>>>>>>>>>> >>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>> The advantage of (1) is that it is simpler. >>>The >>> > > >>> > > >> >>>>>>>>>>>>disadvantage >>> > > >>> > > >> >>>>>>>>>>>>is >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>> >>>>any >>> > > >>> > > >> >>>>>>>>>>>> >>>> member of any group that reads from t can >>>commit >>> > > >>> > > >> >>>>>>>>>>>> >>>>offsets >>> > > >>> > > >> >>>>>>>>>>>>as >>> > > >>> > > >> >>>>>>>>>>>>any >>> > > >>> > > >> >>>>>>>>>>>>other >>> > > >>> > > >> >>>>>>>>>>>> >>>> member of a different group. This doesn't >>>effect >>> > > >>> > > >> >>>>>>>>>>>> >>>> data >>> > > >>> > > >> >>>>>>>>>>>>security >>> > > >>> > > >> >>>>>>>>>>>>(who >>> > > >>> > > >> >>>>>>>>>>>> >>>>can >>> > > >>> > > >> >>>>>>>>>>>> >>>> access what) but it is a bit of a management >>> > > >>> > > >> >>>>>>>>>>>> >>>>issue--a >>> > > >>> > > >> >>>>>>>>>>>>malicious >>> > > >>> > > >> >>>>>>>>>>>>person >>> > > >>> > > >> >>>>>>>>>>>> >>>>can >>> > > >>> > > >> >>>>>>>>>>>> >>>> cause data loss or duplicates for another >>> > > >> >>>>>>>>>>>> >>>> consumer >>> > > >>> > > >> >>>>>>>>>>>> >>>>by >>> > > >>> > > >> >>>>>>>>>>>>committing >>> > > >>> > > >> >>>>>>>>>>>> >>>>offset. >>> > > >>> > > >> >>>>>>>>>>>> >>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>> I think I favor (2) but it's worth it to >>>think >>>it >>> > > >>> > > >> through. >>> > > >>> > > >> >>>>>>>>>>>> >>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>> -Jay >>> > > >>> > > >> >>>>>>>>>>>> >>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>> On Tue, Apr 21, 2015 at 2:43 PM, Parth >>>Brahmbhatt >>> > > >> >>>>>>>>>>>> >>>> < >>> > > >>> > > >> >>>>>>>>>>>> >>>> >>> > > >>> > > >> >>>>>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:pbrahmbhatt@hortonwo >>> > > >> >>>>>>>>>>>>rk >>> > > >>><mailto:pbrahmbh...@hortonworks.com%3cmailto:pbrahmbhatt@hortonwork> >>> > > >>> > > >> >>>>>>>>>>>>s >>> > > >>> > > >> >>>>>>>>>>>>.com >>> > > >>> > > >> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>wrote: >>> > > >>> > > >> >>>>>>>>>>>> >>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> Hey Jun, >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> Yes and we support wild cards for all acl >>> > > >> >>>>>>>>>>>> >>>>> entities >>> > > >>> > > >> >>>>>>>>>>>>principal, >>> > > >>> > > >> >>>>>>>>>>>>hosts >>> > > >>> > > >> >>>>>>>>>>>> >>>>>and >>> > > >>> > > >> >>>>>>>>>>>> >>>>> operation. >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> Thanks >>> > > >>> > > >> >>>>>>>>>>>> >>>>> Parth >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> On 4/21/15, 9:06 AM, "Jun Rao" >>> > > >>> > > >> >>>>>>>>>>>> >>>>>>>><j...@confluent.io<mailto:j...@confluent.io<mailto: >>> > > j...@confluent.io%3cmailto:j...@confluent.io>>> wrote: >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >Harsha, Parth, >>> > > >>> > > >> >>>>>>>>>>>> >>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >Thanks for the clarification. This makes >>>sense. >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >Perhaps >>> > > >>> > > >> >>>>>>>>>>>>we >>> > > >>> > > >> >>>>>>>>>>>>can >>> > > >>> > > >> >>>>>>>>>>>> >>>>>clarify the >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >meaning of those rules in the wiki. >>> > > >>> > > >> >>>>>>>>>>>> >>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >Related to this, it seems that we need to >>> > > >> >>>>>>>>>>>> >>>>> >support >>> > > >>> > > >> >>>>>>>>>>>>wildcard >>> > > >>> > > >> >>>>>>>>>>>>in >>> > > >>> > > >> >>>>>>>>>>>> >>>>>cli/request >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >protocol for topics? >>> > > >>> > > >> >>>>>>>>>>>> >>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >Jun >>> > > >>> > > >> >>>>>>>>>>>> >>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >On Mon, Apr 20, 2015 at 9:07 PM, Parth >>> > > >> >>>>>>>>>>>> >>>>> >Brahmbhatt >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >< >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto<mailto: >>> > > pbrahmbh...@hortonworks.com%3cmailto>: >>> > > >>> > > >> pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>> > > >>> > > >> >>>>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>>wrote: >>> > > >>> > > >> >>>>>>>>>>>> >>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> The iptables on unix supports the DENY >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> operator, not >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>>it >>> > > >>> > > >> >>>>>>>>>>>> >>>>>should >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> matter. The deny operator can also be >>>used >>>to >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> specify >>> > > >>> > > >> >>>>>>>>>>>>³allow >>> > > >>> > > >> >>>>>>>>>>>>user1 >>> > > >>> > > >> >>>>>>>>>>>> >>>>>to >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>READ >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> from topic1 from all hosts but >>>host1,host2². >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>Again we >>> > > >>> > > >> >>>>>>>>>>>>could >>> > > >>> > > >> >>>>>>>>>>>>add a >>> > > >>> > > >> >>>>>>>>>>>> >>>>>host >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> group semantic and extra complexity around >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> that, not >>> > > >>> > > >> >>>>>>>>>>>>sure >>> > > >>> > > >> >>>>>>>>>>>>if >>> > > >>> > > >> >>>>>>>>>>>>its >>> > > >>> > > >> >>>>>>>>>>>> >>>>>worth >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>it. >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> In addition with DENY operator you are now >>> > > >> >>>>>>>>>>>> >>>>> >> not >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>forced >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>>create a >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>special >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> group just to support the authorization >>>use >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>case. I >>> > > >>> > > >> am >>> > > >>> > > >> >>>>>>>>>>>>not >>> > > >>> > > >> >>>>>>>>>>>> >>>>>convinced >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>that >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> the operator it self is really all that >>> > confusing. >>> > > >>> > > >> >>>>>>>>>>>>There >>> > > >>> > > >> >>>>>>>>>>>>are 3 >>> > > >>> > > >> >>>>>>>>>>>> >>>>>practical >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> use cases: >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> - Resource with no acl what so ever -> >>>allow >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> access >>> > > >>> > > >> to >>> > > >>> > > >> >>>>>>>>>>>>everyone ( >>> > > >>> > > >> >>>>>>>>>>>> >>>>>just >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>for >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> backward compatibility, I would much >>>rather >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>fail >>> > > >>> > > >> close >>> > > >>> > > >> >>>>>>>>>>>>and >>> > > >>> > > >> >>>>>>>>>>>>force >>> > > >>> > > >> >>>>>>>>>>>> >>>>>users >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>to >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> explicitly grant acls that allows access >>>to >>> > > >> >>>>>>>>>>>> >>>>> >> all >>> > > >>> > > >> >>>>>>>>>>>>users.) >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> - Resource with some acl attached -> only >>> > > >> >>>>>>>>>>>> >>>>> >> users >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> that >>> > > >>> > > >> >>>>>>>>>>>>have >>> > > >>> > > >> >>>>>>>>>>>>a >>> > > >>> > > >> >>>>>>>>>>>> >>>>>matching >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>allow >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> acl are allowed (i.e. ³allow READ access >>>to >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>topic1 to >>> > > >>> > > >> >>>>>>>>>>>>user1 >>> > > >>> > > >> >>>>>>>>>>>>from >>> > > >>> > > >> >>>>>>>>>>>> >>>>>all >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> hosts², only user1 has READ access and no >>> > > >> >>>>>>>>>>>> >>>>> >> other >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> user >>> > > >>> > > >> >>>>>>>>>>>>has >>> > > >>> > > >> >>>>>>>>>>>>access of >>> > > >>> > > >> >>>>>>>>>>>> >>>>>any >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> kind) >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> - Resource with some allow and some deny >>>acl >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> attached >>> > > >>> > > >> >>>>>>>>>>>>-> >>> > > >>> > > >> >>>>>>>>>>>>users >>> > > >>> > > >> >>>>>>>>>>>>are >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>allowed >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> to perform operation only when they >>>satisfy >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>allow acl >>> > > >>> > > >> >>>>>>>>>>>>and >>> > > >>> > > >> >>>>>>>>>>>>do >>> > > >>> > > >> >>>>>>>>>>>>not >>> > > >>> > > >> >>>>>>>>>>>> >>>>>have >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> conflicting deny acl. Users that have no >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> acl(allow or >>> > > >>> > > >> >>>>>>>>>>>>deny) >>> > > >>> > > >> >>>>>>>>>>>>will >>> > > >>> > > >> >>>>>>>>>>>> >>>>>still >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>not >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> have any access. (i.e. ³allow READ access >>>to >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>topic1 >>> > > >>> > > >> to >>> > > >>> > > >> >>>>>>>>>>>>user1 >>> > > >>> > > >> >>>>>>>>>>>>from >>> > > >>> > > >> >>>>>>>>>>>> >>>>>all >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> hosts except host1 and host², only user1 >>>has >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> access >>> > > >>> > > >> >>>>>>>>>>>>but >>> > > >>> > > >> >>>>>>>>>>>>not >>> > > >>> > > >> >>>>>>>>>>>>from >>> > > >>> > > >> >>>>>>>>>>>> >>>>>host1 >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>an >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> host2) >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> I think we need to make a decision on deny >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> primarily >>> > > >>> > > >> >>>>>>>>>>>>because >>> > > >>> > > >> >>>>>>>>>>>>with >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> introduction of acl management API, Acl is >>> > > >> >>>>>>>>>>>> >>>>> >> now >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> a >>> > > >>> > > >> >>>>>>>>>>>>public >>> > > >>> > > >> >>>>>>>>>>>>class >>> > > >>> > > >> >>>>>>>>>>>>that >>> > > >>> > > >> >>>>>>>>>>>> >>>>>will >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>be >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> used by Ranger/Santry and other >>>authroization >>> > > >>> > > >> >>>>>>>>>>>>providers. >>> > > >>> > > >> >>>>>>>>>>>>In >>> > > >>> > > >> >>>>>>>>>>>> >>>>>Current >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>design >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> the acl has a permissionType enum field >>>with >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>possible >>> > > >>> > > >> >>>>>>>>>>>>values >>> > > >>> > > >> >>>>>>>>>>>>of >>> > > >>> > > >> >>>>>>>>>>>> >>>>>Allow >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>and >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> Deny. If we chose to remove deny we can >>> > > >> >>>>>>>>>>>> >>>>> >> assume >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>all >>> > > >>> > > >> >>>>>>>>>>>>acls >>> > > >>> > > >> >>>>>>>>>>>>to >>> > > >>> > > >> >>>>>>>>>>>>be >>> > > >>> > > >> >>>>>>>>>>>>of >>> > > >>> > > >> >>>>>>>>>>>> >>>>>allow >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> type and remove the permissionType field >>> > > >>> > > >>completely. >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> Thanks >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> Parth >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> On 4/20/15, 6:12 PM, "Gwen Shapira" >>> > > >>> > > >> >>>>>>>>>>>> >>>>>>>><gshap...@cloudera.com<mailto:gshapira@cloudera.c >>> > > >> >>>>>>>>>>>> >>>>>om >>> > > >>> > > >> >>>>>>>>>>>> >>>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>>> >>> > > >>> > > >> >>>>>>>>>>>>wrote: >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >I think thats how its done in pretty much >>> > > >> >>>>>>>>>>>> >>>>> >> >any >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >system >>> > > >>> > > >> >>>>>>>>>>>>I >>> > > >>> > > >> >>>>>>>>>>>>can >>> > > >>> > > >> >>>>>>>>>>>>think >>> > > >>> > > >> >>>>>>>>>>>> >>>>>of. >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> > >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>>>> >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> >> >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> > >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>> >>> > > >>> > > >> >>>>>>>>>>>-- >>> > > >>> > > >> >>>>>>>>>>>Jeff Holoman >>> > > >>> > > >> >>>>>>>>>>>Systems Engineer >>> > > >>> > > >> >>>>>>>>>> >>> > > >>> > > >> >>>>>>>>> >>> > > >>> > > >> >>>>>>> >>> > > >>> > > >> >>>>> >>> > > >>> > > >> >>> >>> > > >>> > > >> > >>> > > >>> > > >> >>> > > >>> > > >>> > > >>> > >>> >
