Re: [VOTE] KIP-11- Authorization design for kafka security

Parth Brahmbhatt Thu, 30 Apr 2015 12:16:20 -0700

Hi Joe,

Regarding open question: I changed the title to “Questions resolved after
community discussions” let me know if you have a better name. I have a
question and a bullet point under each question describing the final
decision. Not sure how can I make it any cleaner so appreciate any
suggestion.


Regarding system tests: I went through a bunch of KIP none of which
mentions what test cases will be added. Do you want to add a “How do you
plan to tet” section in the general KIP template or you think this is
just a special case where the test cases should be listed and discussed as
part of KIP? I am not sure if KIP really is the right forum for this
discussion. This can easily be addressed during code review if people
think we don’t have enough test coverage.

I am still not sure which part is not clear. The scala exception is added
for internal server side rpresentation. In the end all of our responses
always return just an error code for which we will add an
AuthorizationErroCode mapped to AuthorizationException. The error code it
self will not reveal any information other then the fact that you are not
authorized to perform an operation on a resource and you will get this
error code even for non existent topics if no acls exist for those topics.

 can add a diagram if that makes things more clear, I am not convinced
its needed given we have come so far without it. Essentially there are 3
steps
        * users use the acl cli to add acls to their topics/groups/cluster
        * brokers start with a broker config that specifies what authorizer
implementation to use.
        * every api request first goes through the authorizer and fails if
authorizer denies it. (authorizer implementation described in the doc with
pseudo code)

Note: Authentication/Wire Encryption is a separate piece and is being
discussed actively in another KIP if that is the detail you are looking
for.

I think the description under this section
https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+In
terface#KIP-11-AuthorizationInterface-DataFlows captures the internal
details.

Thanks
Parth

On 4/30/15, 11:24 AM, "Joe Stein" <joe.st...@stealth.ly> wrote:

>Gwen << regarding additional authorizers
>
>I think having these in the system tests duals as both good confidence in
>language independency of the changes. It also makes sure that when we
>release that we don't go breaking Sentry or Ranger or anyone else that
>wants to integrate.
>
>Gwen << Regarding "AuthorizationException
>
>Yeah so I have two issues. The one you raised yes, 100%. Also I don't
>unerstand how that is not a broker wire protocol response and only a JVM
>exception.
>
>Jun << Could you elaborate on why we should not store JSON in ZK? So far,
>all existing ZK data are in JSON.
>
>If I have 1,000,000 users in LDAP and 150 get access to Kafka topics
>through this mechanism then I have to go and parse and push all of my
>changes into zookeeper for it to take affect?
>
>If someone wanted to implement SAML I don't think this would work. Not
>sure
>how it would work with NiFi either (something around here I think maybe
>https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;a=blob;f=nar-
>bundles/framework-bundle/framework/web/web-security/src/main/java/org/apac
>he/nifi/web/security/authorization/NiFiAuthorizationService.java;hb=e67eb4
>f5>).
>
>Parth << All the open issues already have a resolution , I can open a jira
>for each one and add the resolution to it and resolve them immediately if
>you want thisfor tracking purposes.
>
>Are those inline to the question with the <li> <li> I didn't quite get
>that
>section at all. If the open questions are answered then they aren't open
>can you tidy that up then.
>
>Parth <<  We will update system tests to verify that the code works. We
>have thorough unit tests for all the new code except for modificaions
>made
>to KafkaAPI as that has way too many dependencies to be mocked which I
>guess is the reason for no existing unit tests.
>
>Can you update the KIP with some more detail about that please.
>
>Parth << I don’t know if I completely understand the concern. We have
>talked with Ranger team (Don Bosco Durai) so we at least have one custom
>authorizer implementation that as approved this design and they will be
>able to inject their authorization framework with current interfaces. Do
>you see any issue with the design which will prevent anyone frm providing
>a custom implementation?
>
>Maybe a diagram for all of the different parts interacting. I still don't
>get why there are no wire protocol changes and just change in the JVM.
>What
>do non-jvm clients do and how do they work with Kafka. Very confusing,
>almost obfuscating.
>
>~ Joestein
>
>
>On Thu, Apr 30, 2015 at 1:14 PM, Gwen Shapira <gshap...@cloudera.com>
>wrote:
>
>> * Regarding additional authorizers:
>> Prasad, who is a PMC on Apache Sentry reviewed the design and confirmed
> Sentry can integrate with the current APIs. Dapeng Sun, a committer on
>> Sentry had some concerns about the IP privileges and how we prioritize
>> privileges - but nothing that prevents Sentry from integrating with the
>> existing solution, from what I could see. It seems to me that the
>>design is
>> very generic and adapters can be written for other authorization systems
>> (after all, you just need to implement setACL, getACL and Authorize -
>>all
>> pretty basic), although I can't speak for Oracle's Identity Manager
>> specifically.
>>
>> * Regarding "AuthorizationException to indicate that an operation was
>>not
>> authorized": Sorry I missed this in previous reviewed, but now that I
>>look
>> at it - Many systems intentionally don't return AuthorizationException
>>when
>> READ privilege is missing, since this already gives too much information
>> (that the topic exists and that you don't have privileges on it).
>>Instead
>> they return a variant of "doesn't exist". I'm wondering if this
>>approach is
>> applicable / desirable for Kafka as well.
>> Note that this doesn't remove the need for AuthorizationException - I'm
>> just suggesting a possible refinement on its use.
>>
>> Gwen
>>
>>
>>
>> On Thu, Apr 30, 2015 at 9:52 AM, Parth Brahmbhatt <
>> pbrahmbh...@hortonworks.com> wrote:
>>
>> > Hi Joe, Thanks for taking the time to review.
>> >
>> > * All the open issues already have a resolution , I can open a jira
>>for
>> > each one and add the resolution to it and resolve them immediately if
>>you
>> > want this for tracking purposes.
>> > * We will update system tests to verify that the code works. We have
>> > thorough unit tests for all the new code except for modifications
>>made to
>> > KafkaAPI as that has watoo many dependencies to be mocked which I
>>guess
>> > is the reason for no existing unit tests.
>> > * I don’t know if I completely understand the concern. We have talked
>> with
>> > Ranger team (Don Bosco Durai) so we at least have one custom
>>authorizer
>> > implementation that has approved this design and they will be able to
>> > inject their authorization framewrk with current interfaces. Do you
>>see
>> > any issue with the design which will prevent anyone from providing a
>> > custom implementation?
>> > * Did not understand the concern around wire protocol, we are adding
>> > AuthorizationException to indicate that an operation was not
>>authorized.
>> >
>> > Thanks
>> > Parth
>> >
>> > On 4/30/15, 5:59 AM, "Jun Rao" <j...@confluent.io> wrote:
>> >
>> > >Joe,
>> > >
>> > >Could you elaborate on why we should not store JSON in ZK? So far,
>>all
>> > >existing ZK data are in JSON.
>> > >
>> > >Thanks,
>> > >
>> > >Jun
>> > >
>> > >On Thu, Apr 30, 2015 at 2:06 AM, Joe Stein <joe.st...@stealth.ly>
>> wrote:
>> > >
>> > >> Hi, sorry I am coming in late to chime back in on this thread and
>> > >>haven't
>> > >> been able to make the KIP hangouts the last few weeks. Sorry if
>>any of
>> > >>this
>> > >> wasbrought up already or I missed it.
>> > >>
>> > >> I read through the KIP and the thread(s) and a couple of things
>>jumped
>> > >>out.
>> > >>
>> > >>
>> > >>    - Can we break out the open issues in JIRA (maybe during the
>> hangout)
>> > >>    that are in the KIP and resolve/flesh those out more?
>> > >>
>> > >>
>> > >>
>> > >>    - I don't see any updates with the systems test or how we can
>>know
>> > >>the
>> > >>    code works.
>> > >>
>> > >>
>> > >>
>> > >>    - We need some implementation/example/sample that we know can
>>work
>> in
>> > >>    all different existing entitlement servers and not just ones
>>that
>> > >>run in
>> > >>    types of data centers too. I am not saying we should support
>> > >>everything
>> > >> but
>> > >>    if someone had to implement
>> > >>    https://docs.oracle.com/cd/E19225-01/820-6551/bzafm/index.html
>> with
>> > >>    Kafka it has to work for them out of the box.
>> > >>
>> > >>
>> > >>
>> > >>    - We should shy away from storing JSON in Zookeeper. Lets store
>> > >>bytes in
>> > >>    Storage.
>> > >>
>> > >>
>> > >>
>> > >>    - We should spend some time thinking through exceptions in the
>>wire
>> > >>    protocol maybe as part of this so it can keep moving forward.
>> > >>
>> > >>
>> > >> ~ Joe Stein
>> > >>
>> > >> On Tue, Apr 28, 2015 at 3:33 AM, Sun, Dapeng <dapeng....@intel.com>
>> > >>wrote:
>> > >>
>> > >> > Thank you for your reply, Gwen.
>> > >> >
>> > >> > >1. Complex rule systems can be difficult to reason about and
>> > >>therefore
>> > >> > end up being less secure. The rule "Deny always wins" is very
>>easy
>> to
>> > >> grasp.
>> > >> > Yes, I'm agreed with your point: we should not make the rule
>> complex.
>> > >> >
>> > >> > >2. We currently don't have any mechanism for specifying IP
>>ranges
>> (or
>> > >> host
>> > >> > >ranges) at all. I think its a pretty significant deficiency,
>>but it
>> > >>does
>> > >> > mean that we don't need to worry about the issue of blocking a
>>large
>> > >> range
>> > >> > while unblocking few servers in the range.
>> > >> > Support ranges sounds reasonable. If this feature will be in
>> > >>development
>> > >> > plan, I also don't think we can put "the best matching acl" and "
>> > >>Support
>> > >> > ip ranges" together.
>> > >> >
>> > >> > >We have a call tomorrow (Tuesday, April 28) at 3pm PST - to
>>discuss
>> > >>this
>> > >> > and other outstanding design issues (not all related to
>>security).
>> If
>> > >>you
>> > >> > are interested in joining - let me know and I'll forward you the
>> > >>invite.
>> > >> > Thank you, Gwen. I have the invite and I should be at home at
>>that
>> > >>time.
>> > >> > But due to network issue, I may can't join the meeting smoothly.
>> > >> >
>> > >> > Regards
>> > >> > Dapeng
>> > >> >
>> > >> > -----Original Message-----
>> > >> > From: Gwen Shapira [mailto:gshap...@cloudera.com]
>> > >> > Sent: Tuesday, April 28, 2015 1:31 PM
>> > >> > To: dev@kafka.apache.org
>> > >> > Subject: Re [VOTE] KIP-11- Authorization design for kafka
>>security
>> > >> >
>> > >> > While I see the advantage of being able to say something like:
>>"deny
>> > >>user
>> > >> > X from hosts h1...h200" also "allow user X from host h189", there
>> are
>> > >>two
>> > >> > issues here:
>> > >> >
>> > >> > 1. Complex rule systems can be difficult to reason about and
>> therefore
>> > >> end
>> > >> > up being less secure. The rule "Deny always wins" is very easy to
>> > >>grasp.
>> > >> >
>> > >> > 2. We currently don't have any mechanism for specifying IP ranges
>> (or
>> > >> host
>> > >> > ranges) at all. I think its a pretty significant deficiency, but
>>it
>> > >>does
>> > >> > mean that we don't need to worry about the issue of blocking a
>>large
>> > >> range
>> > >> > while unblocking few servers in the range.
>> > >> >
>> > >> > Gwen
>> > >> >
>> > >> > P.S
>> > >> > We have a call tomorrow (Tuesday, April 28) at 3pm PST - to
>>discuss
>> > >>this
>> > >> > and other outstanding design issues (not all related to
>>security).
>> If
>> > >>you
>> > >> > are interested in joining - let me know and I'll forward you the
>> > >>invite.
>> > >> >
>> > >> > Gwen
>> > >> >
>> > >> > On Mon, Apr 27, 2015 at 10:15 PM, Sun, Dapeng
>><dapeng....@intel.com
>> >
>> > >> > wrote:
>> > >> >
>> > >> > > Attach the image.
>> > >> > >
>> > >> > >
>> > 
>>>>https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-ac
>> > >> > > l1.png
>> > >> > >
>> > >> > > Regards
>> > >> > > Dapeng
>> > >> > >
>> > >> > > From: Sun, Dapeng [mailto:dapeng....@intel.com]
>> > >> > > Sent: Tuesday, April 28, 2015 11:44 AM
>> > >> > > To: dev@kafka.apache.org
>> > >> > > Subject: RE: [VOTE] KIP-11- Authorization design for kafka
>> security
>> > >> > >
>> > >> > >
>> > >> > > Thank you for your rapid reply, Parth.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > >* I think the wiki already describes the precedence order as
>>Deny
>> > >> > > >taking
>> > >> > > precedence over allow when conflicting acls are found
>> > >> > >
>> > 
>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizati
>> > >> > > on+In
>> > >> > >
>> > >> > > >terface#KIP-11-AuthorizationInterface-PermissionType
>> > >> > >
>> > >> > > Got it, thank you.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > >* In the first version that I am currently writing there is no
>> > >>group
>> > >> > > support. Even when we add it I don't see the need to add a
>> > >>precedence
>> > >> > > for evaluation. it does not matter which principal matches as
>>long
>> > >>as
>> > >> > >
>> > >> > > > we have a match.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > About this part, I think we should choose the best matching acl
>> for
>> > >> > > authorization, no mater we support group or not.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > For the case
>> > >> > >
>> > >> > >  [cid:image001.png@01D08197.E94BD410]
>> > >> > >
>> > >> > >
>> > 
>>>>https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-ac
>> > >> > > l1.png
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > if 2 Acls are define, one that deny an operation from all
>>hosts
>> and
>> > >> > > one that allows the operation from host1, the operation from
>>host1
>> > >> > > will be denied or allowed?
>> > >> > >
>> > >> > > According wiki "Deny will take precedence over Allow in
>>competing
>> > >> > > acls.", it seems acl_1 will win the competition, but customers'
>> > >> > > intention may be "allow".
>> > >> > >
>> > >> > > I think "deny always take precedence over Allow" is okay, but
>> > >>"host1
>> > >> > > -> user1"  >  "host1 "  >  "default" may make sense.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > >* Acl storage is indexed by resource right now because that is
>> the
>> > >> > > primary lookup id for all authorize operations. Given acls are
>> > >>cached
>> > >> > > I don't see the need to optimized the storage layer any further
>> for
>> > >> > lookup.
>> > >> > >
>> > >> > > >* The reason why we have acl with multi everything is to
>>reduce
>> > >> > > redundancy in acl storage. I am not sure how wil we be able to
>> > >>reduce
>> > >> > > redundancy if we divide it by using one principal,one host, one
>> > >> > operation.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > Yes, I'm also greed with "Acl storage should be indexed by
>> > >>resource".
>> > >> > > Under resource index, it may be better to add index such as
>>hosts
>> > >>and
>> > >> > > principals. One option may be one principal, one host, one
>> > >>operation.
>> > >> > > Just give your these scenarios for considering.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > For the case defined in wiki:>> > >> > >
>> > >> > > Acl_1 -> {"user:bob", "user:*"} is allowed to READ from all
>>hosts.
>> > >> > >
>> > >> > > Acl_2 -> {"user:bob"} is denied to READ from host1
>> > >> > >
>> > >> > > Acl_3 -> {"user:alice", "group:kafka-devs"} is allowed to READ
>>and
>> > >> > > WRITE from {host1, hos2}.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > For acl_3, if we want to remove alice's WRITE from
>>{host1,host2}
>> and
>> > >> > > remove alice's READ from host1, user may have following ways to
>> > >> achieve:
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > 1.Remove the parts of acl_3 directly, I think if we make it
>> divided
>> > >> > > and hierarchical, this kind of operatons could be done
>>directly
>> in
>> > >> > backend.
>> > >> > >
>> > >> > > 2.Remove acl_3, and add new acl {"group:kafka-devs"} is
>>allowed to
>> > >> > > READ and WRITE from {host1, host2} and {"user:alice" } is
>>allowed
>> to
>> > >> > > READ from {host2}
>> > >> > >
>> > >> > > 3.Add two denied acls,{ user:alice} is denied to WRITE from
>> > >> > > {host1,host2} and { user:alice} is denied to READ from {host1}
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > All these can achieve this kind of operations, but I think 1
>>could
>> > >> > > more directly for user operations. If you think this
>>optimization
>> is
>> > >> > > not urgent, I'm also agreed.
>> > >> > >
>>  >> > >
>> > >> > >
>> > >> > > Regards
>> > >> > >
>> > >> > > Dapeng
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > -----Original Message-----
>> > >> > >
>> > >> > > From: Parth Brahmbhatt [mailto:pbrahmbh...@hortonworks.com]
>> > >> > >
>> > >> > > Sent: Tuesday, April 28, 2015 12:18 AM
>> > >> > >
>> > >> > > To: dev@kafka.apache.org<milto:dev@kafka.apache.org>
>> > >> > >
>> > >> > > Subject: Re: [VOTE] KIP-11- Authorization design for kafka
>> security
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > Hi Sun, thanks for the comments, my answers are below:
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > * I think the wiki already describes the precedence oder as
>>Deny
>> > >> > > taking precedence over allow when conflicting acls are found
>> > >> > >
>> > 
>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizati
>> > >> > > on+In
>> > >> > >
>> > >> > > terface#KIP-11-AuthorizationInterface-PermissionType
>> > >> > >
>> > >> > > * In the first version that I am currently writing there is no
>> group
>> > >> > > support. Even when we add it I don't see the need to add a
>> > >>precedence
>> > >> > > for evaluation. it does not matter which principal matches as
>>long
>> > >>as
>> > >> > > we have a match.
>> > >> > >
>> > >> > > * Acl storage is indexed by resource right now because that is
>>the
>> > >> > > primary lookup id for all authorize operations. Given acl are
>> > >>cached
>> > >> > > I don't see the need to optimized the storage layer any further
>> for
>> > >> > lookup.
>> > >> > >
>> > >> > > * The reason why we have acl with multi everything is to reduce
>> > >> > > redundancy in acl storage. I am not sure how will we be able to
>> > >>reduce
>> > >> > > redundancy if we divide it by using one principal,one host, one
>> > >> > operation.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > Thanks
>> > >> > >
>> > >> > > Parth
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > On 4/26/15, 8:06 PM, "Sun, Dapeng"
>><dapeng....@intel.com<mailto:
>> > >> > > dapeng....@intel.com>> wrote:
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > >Hi Parth
>> > >> > >
>> > >> > > >
>> >>> > >
>> > >> > > >The design looks good, a few minor comments below. Since I
>>just
>> > >> > > >started
>> > >> > >
>> > >> > > >looking into the discussion and many previous discussions I
>>may
>> > >> > > >missed,
>> > >> > >
>> > >> > > >I'm sorry if these comments had be discussed.
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >1. About SimleAclAuthorizer (SimpleAuthorizer):
>> > >> > >
>> > >> > > >a. As my understanding, I think there should only one type
>> > >> > >
>> > >> > > >privilege(allow/deny) of a topic on a principle, or we make it
>> > >>deny >
>> > >> > >
>> > >> > > >allow.
>> > >> > >
>> > >> > > >For example, acl_1 " host1 -> group1-> user1 -> read->allow"
>>and
>> > >> acl_2 "
>> > >> > >
>> > >> > > >host1-> group1 -> usr1 ->read->deny", if the two acls are
>>for a
>> > >>same
>> > >> > >
>> > >> > > >topic, it may be hard to understand, do you think it's
>>necessary
>> to
>> > >> > > >add
>> > >> > >
>> > >> > > >some details about this to wiki.
>> > >> > >
>> > >> > > >b. And when we do authorize a user on a topic, we may should
>> check
>> > >> > >
>> > >> > > >user's user level acl first, then check user's group level
>>acl,
>> > >> > > >finally
>> > >> > >
>> > >> > > >we check the host level and default level acl. do you think
>>it's
>> > >> > >
>> > >> > > >necessary we add some contents like these to wiki.
>> > >> > >
>> > >> > > >For example, "host1 -> group1-> user1"  >  "host1 -> group1"
>>>
>> > >> "host1"
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >2.About SimpleAclAuthorizer (Acl Json will be stored in
>> zookeeper)
>> > >>a.
>> > >> > >
>> > >> > > >It may be better to make acl json stored hierarchily. It may
>>be
>> > >>easy
>> > >> > > >to
>> > >> > >
>> > >> > > >search and do authorize. For example, when we authorize a
>>user,
>> we
>> > >> > > >only
>> > >> > >
>> > >> > > >need user related acls.
>> > >> > >
>> > >> > > >b. I found one acl may contains multi-principles,
>> multi-operations
>> > >> > > >and
>> > >> > >
>> > >> > > >multi-hosts, I'm strongly agreed with we provide api like
>>these,
>> > >>but
>> > >> > >
>> > >> > > >the acls stored in zookeeper or memory we may better to
>>separate
>> to
>> > >> > >
>> > >> > > >one-principle, one-operation and one host. So we could make
>>sure
>> > >> > > >there
>> > >> > >
>> > >> > > >are not many acls with same meaning and make acl management
>> easily.
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >Regards
>> > >> > >
>> > >> > > >Dapeng
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >-----Original Message-----
>> > >> > >
>> > >> > > >From: Jun Rao [mailto:j...@confluent.io]
>> > >> > >
>> > >> > > >Sent: Monday, April 27, 2015 5:02 AM
>> > >> > >
>> > >> > > >To: dev@kafka.apache.org<mailto:dev@kafka.apache.org>
>> > >> > >
>> > >> > > >Subject: Re: [VOTE] KIP-11- Authorization design for kafka
>> security
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >A few more minor comments.
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >100. To make it clear, perhaps we should rename the resource
>> > >>"group"
>> > >> > > >to
>> > >> > >
>> > >> > > >consumer-group. We can probably make the same change in CLI as
>> well
>> > >> > > >so
>> > >> > >
>> > >> > > >that it's not confused with user group.
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >101. Currently, create is only at the cluster level. Should it
>> also
>> > >> > > >be
>> > >> > >
>> > >> > > >at topic level? For example, perhaps it's useful to allow only
>> > >>user X
>> > >> > >
>> > >> > > >to create topic X.
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >Thanks,
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >Jun
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >On Sun, Apr 26, 2015 at 12:36 AM, Gwen Shapira
>> > >><gshap...@cloudera.com
>> > >> > > <mailto:gshap...@cloudera.com>>
>> > >> > >
>> > >> > > >wrote:
>> > >> > >
>> > >> > > >
>> > >> > >
>> > >> > > >> Thanks for clarifying, Parth. I think you are taking the
>>right
>> > >> > >
>> > >> > > >> approach here.
>> > >> > >
>> > >> > > >>
>> > >> > >
>> > >> > > >> On Fri, Apr 24, 2015 at 11:46 AM, Parth Brahmbhatt
>> > >> > >
>> > >> > > >> <pbrahmbh...@hortonworks.com<mailto:
>> pbrahmbh...@hortonworks.com
>> > >>
>> > >> > > wrote:
>> > >> > >
>> > >> > > >> > Sorry Gwen, completely misunderstood the question :-).
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> > * Does everyone have the privilege to create a new Group
>>and
>> > >>use
>> > >> > > >> > it
>> > >> > >
>> > >> > > >> > to consume from Topics he's already privileged on?
>> > >> > >
>> > >> > > >> >         Yes in current proposal. I did not see an API to
>> create
>> > >> > >
>> > >> > > >> > group
>> > >> > >
>> > >> > > >> but if you
>> > >> > >
>> > >> > > >> > have a READ permission on a TOPIC and WRITE permission on
>> that
>> > >> > >
>> > >> > > >> > Group you are free to join and consume.
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> > * Will the CLI tool be used to manage group membership
>>too?
>> > >> > >
>> > >> > > >> >         Yes and I think that means I need to add ―group.
>> > >>Updating
>> > >> > >
>> > >> > > >> > the
>> > >> > >
>> > >> > > >> KIP. Thanks
>> > >> > >
>> > >> > > >> > for pointing this out.
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> > * Groups are kind of ephemeral, right? If all consumers in
>> the
>> > >> > >
>> > >> > > >> > group disconnect the group is gone, AFAIK. Do we preserve
>>the
>> > >> ACLs?
>> > >> > >
>> > >> > > >> > Or do we treat the new group as completely new resource?
>>Can
>> we
>> > >> > >
>> > >> > > >> > create ACLs before the group exists, in anticipation of it
>> > >> > > >> > getting
>> > >> > > created?
>> > >> > >
>> > >> > > >> >         I have considered any auto delete and auto create
>>as
>> > >>out
>> > >> > > >> > of
>> > >> > >
>> > >> > > >> scope for the
>> > >> > >
>> > >> > > >> > first release. So Right now I was going with preserving
>>the
>> > >>acls.
>> > >> > >
>> > >> > > >> > Do you see any issues with this? Auto deleting would mean
>> > >> > >
>> > >> > > >> > authorizer will now have to get into implementation
>>details
>> of
>> > >> > >
>> > >> > > >> > kafka which I was trying to avoid.
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> > Thanks
>> > >> > >
>> > >> > > >> > Parth
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> > On 4/24/15, 11:33 AM, "Gwen Shapira"
>><gshap...@cloudera.com
>> > >> <mailto:
>> > >> > > gshap...@cloudera.com>> wrote:
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >> >>We are not talking about same Groups :)
>> > >> > >
>> > >> > > >> >>
>> > >> > >
>> > >> > > >> >>I meant, Groups of consumers (which KIP-11 lists as a
>> separate
>> > >> > >
>> > >> > > >> >>resource in the Privilege table)
>> > >> > >
>> > >> > > >> >>
>> > >> > >
>> > >> > > >> >>On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt
>> > >> > >
>> > >> > > >>
>> > >>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>>
>> > >> > > wrote:
>> > >> > >
>> > >> > > >> >>> I see Groups as something we can add incrementally in
>>the
>> > >> > > >> >>> current
>> > >> > >
>> > >> > > >> model.
>> > >> > >
>> > >> > > >> >>> The acls take principalType: name so groups can be
>> > >>represented
>> > >> > > >> >>> as
>> > >> > >
>> > >> > > >> group:
>> > >> > >
>> > >> > > >> >>> groupName. We are not managing group memberships
>>anywhere
>> in
>> > >> > >
>> > >> > > >> >>> kafka and
>> > >> > >
>> > >> > > >> I
>> > >> > >
>> > >> > > >> >>> don't see the need to do so.
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >>> So for a topic1 using the CLI an admin can add an acl to
>> > >>grant
>> > >> > >
>> > >> > > >> >>> access
>> > >> > >
>> > >> > > >> to
>> > >> > >
>> > >> > > >> >>> group:kafka-test-users.
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >>> The authorizer implementation can have a plugin to map
>> > >> > >
>> > >> > > >> >>>authenticated user  to groups ( This is how hadoop and
>>storm
>> > >> > >
>> > >> > > >> >>>works). The plugin could be  mapping user to
>> linux/ldap/active
>> > >> > >
>> > >> > > >> >>>directory groups but that is again upto  the
>>implementation.
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >>> What we are offering is an interface that is extensible
>>so
>> > >> > > >> >>> these
>> > >> > >
>> > >> > > >> >>>features  can be added incrementally. I can add support
>>for
>> > >>this
>> > >> > >
>> > >> > > >> >>>in the first  release but don't necessarily see why this
>> would
>> > >> > > >> >>>be
>> > >> > >
>> > >> > > >> >>>absolute necessity.
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >>> Thanks
>> > >> > >
>> > >> > > >> >>> Parth
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >>> On 4/24/15, 11:00 AM, "Gwen Shapira" <
>> gshap...@cloudera.com
>> > >> > <mailto:
>> > >> > > gshap...@cloudera.com>> wrote:
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >>>>Thanks.
>> > >> > >
>> > >> > > >> >>>>
>> > >> > >
>> > >> > > >> >>>>One more thing I'm missing in the KIP is details on the
>> Group
>> > >> > >
>> > >> > > >> >>>>resource (I think we discussed this and it was just not
>> fully
>> > >> > >
>> > >> > > >>updated):
>> > >> > >
>> > >> > > >> >>>>
>> > >> > >
>> > >> > > >> >>>>* Does everyone have the privilege to create a new Group
>> and
>> > >> > > >> >>>>use
>> > >> > >
>> > >> > > >> >>>>it to consume from Topics he's already privileged on?
>> > >> > >
>> > >> > > >> >>>>* Will the CLI tool be used to manage group membership
>>too?
>> > >> > >
>> > >> > > >> >>>>* Groups are kind of ephemeral, right? If all consumers
>>in
>> > >>the
>> > >> > >
>> > >> > > >> >>>>group disconnect the group is gone, AFAIK. Do we
>>preserve
>> the
>> > >> > >
>> > >> > > >> >>>>ACLs? Or do we treat the new group as completely new
>> > >>resource?
>> > >> > >
>> > >> > > >> >>>>Can we create ACLs before the group exists, in
>>anticipation
>> > >>of
>> > >> > > >> >>>>it
>> > >> > >
>> > >> > > >>getting created?
>> > >> > >
>> > >> > > >> >>>>
>> > >> > >
>> > >> > > >> >>>>Its all small details, but it will be difficult to
>> implement
>> > >> > >
>> > >> > > >> >>>>KIP-11 without knowing the answers :)
>> > >> > >
>> > >> > > >> >>>>
>> > >> > >
>> > >> > > >> >>>>Gwen
>> > >> > >
>> > >> > > >> >>>>
>> > >> > >
>> > >> > > >> >>>>
>> > >> > >
>> > >> > > >> >>>>On Fri, Apr 24, 2015 at 9:58 AM, Parth Brahmbhatt
>> > >> > >
>> > >> > > >>
>> > >>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
>> > >> > > >> >>>>>>
>> > >> > > wrote:
>> > >> > >
>> > >> > > >> >>>>> You are right, moved it to the default implementation
>> > >>section.
>> > >> > >
>> > >> > > >> >>>>>
>> > >> > >
>> > >> > > >> >>>>> Thanks
>> > >> > >
>> > >> > > >> >>>>> Parth
>> > >> > >
>> > >> > > >> >>>>>
>> > >> > >
>> > >> > > >> >>>>> On 4/24/15, 9:52 AM, "Gwen Shapira" <
>> gshap...@cloudera.com
>> > >> > > <mailto:gshap...@cloudera.com>> wrote:
>> > >> > >
>> > >> > > >> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>Sample ACL JSON and Zookeeper is in public API, but I
>> > >>thought
>> > >> > >
>> > >> > > >> >>>>>>it is part of DefaultAuthorizer (Since Sentry and
>>Argus
>> > >>won't
>> > >> > >
>> > >> > > >> >>>>>>be using Zookeeper).
>> > >> > >
>> > >> > > >> >>>>>>Am I wrong? Or is it the KIP?
>> > >> > >
>> > >> > > >> >>>>>>
>> > >> > >
>> > >> > > >> >>>>>>On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt
>> > >> > >
>> > >> > > >>
>> > >>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbhatt@hortonworks.c
>> > >> > > >> >>>>>>om>>
>> > >> > > wrote:
>> > >> > >
>> > >> > > >> >>>>>>> Thanks for clarifying Gwen, KIP updated.
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>> I tried to make the distinction by creating a
>>section
>> for
>> > >> > > >> >>>>>>> all
>> > >> > >
>> > >> > > >> public
>> > >> > >
>> > >> > > >> >>>>>>>APIs
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >>
>> > >>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriz
>> > >> > > >> at
>> > >> > >
>> > >> > > >> >>>>>>>io
>> > >> > >
>> > >> > > >> >>>>>>>n+
>> > >> > >
>> > >> > > >> >>>>>>>In
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >>
>> > >>>>>>>>>terface#KIP-11-AuthorizationInterface-PublicInterfacesandcla
>> > >> > > >> >>>>>>>ss
>> > >> > >
>> > >> > > >> >>>>>>>e
>> > >> > >
>> > >> > > >> >>>>>>>s
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>> Let me know if you think there is a better way to
>> reflect
>> > >> > this.
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>> Thanks
>> > >> > >
>> > >> > > >> >>>>>>> Parth
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>> On 4/24/15, 9:37 AM, "Gwen Shapira"
>> > >><gshap...@cloudera.com
>> > >> > > <mailto:gshap...@cloudera.com>>
>> > >> > >
>> > >> > > >>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>+1 (non-binding)
>> > >> > >
>> > >> > > >> >>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>Two nitpicks for the wiki:
>> > >> > >
>> > >> > > >> >>>>>>>>* Heartbeat is probably a READ and not CLUSTER
>> operation.
>> > >> > > >> >>>>>>>>I'm
>> > >> > >
>> > >> > > >> pretty
>> > >> > >
>> > >> > > >> >>>>>>>>sure new consumers need it to be part of a consumer
>> > >>group.
>> > >> > >
>> > >> > > >> >>>>>>>>* Can you clearly separate which parts are the API
>> > >>(common
>> > >> > > >> >>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>every
>> > >> > >
>> > >> > > >> >>>>>>>>Authorizer) and which parts are DefaultAuthorizer
>> > >> > >
>> > >> > > >>implementation?
>> > >> > >
>> > >> > > >> It
>> > >> > >
>> > >> > > >> >>>>>>>>will make reviews and Authorizer implementations a 
>>bit
>> > >> > > >> >>>>>>>>easier
>> > >> > >
>> > >> > > >> >>>>>>>>to know exactly which is which.
>> > >> > >
>> > >> > > >> >>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>Gwen
>> > >> > >
>> > >> > > >> >>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>On Fri, Apr 24, 2015 at 9:28 AM, Parth Brahmbhatt
>> > >> > >
>> > >> > > >>
>> > >>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbhatt@hortonworks
>> > >> > > >> >>>>>>>>.com>>
>> > >> > > wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>> Hi,
>> > >> > >
>> > >> > > >> >>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>> I would like to open KIP-11 for voting.
>> > >> > >
>> > >> > > >> >>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>> Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>> Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>> On 4/22/15, 1:56 PM, "Parth Brahmbhatt"
>> > >> > >
>> > >> > > >> >>>>>>>>><pbrahmbh...@hortonworks.com<mailto:
>> > >> > > pbrahmbh...@hortonworks.com>>
>> > >> > >
>> > >> > > >> >>>>>>>>> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>Hi Jeff,
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>Thanks a lot for the review. I think you have a 
>>valid
>> > >> > > >> >>>>>>>>>>point
>> > >> > >
>> > >> > > >> >>>>>>>>>>about acls being duplicated and the simplest 
>>solution
>> > >> > > >> >>>>>>>>>>would
>> > >> > >
>> > >> > > >> >>>>>>>>>>be to modify
>> > >> > >
>> > >> > > >> acls
>> > >> > >
>> > >> > > >> >>>>>>>>>>class
>> > >> > >
>> > >> > > >> >>>>>>>>>>so they hold a set of principals instead of single
>> > >> > >
>> > >> > > >> >>>>>>>>>>principal. i.e
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>><user_a,user_b> has <READ,WRITE,DESCRIBE> 
>>Permissions
>> > >>on
>> > >> > >
>> > >> > > >> >>>>>>>>>><Topic1> from <Host1, Host2, Host3>.
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>I think the evaluation order only matters for the
>> > >> > >
>> > >> > > >> >>>>>>>>>>permissionType which is Deny acls should be 
>>evaluated
>> > >> > >
>> > >> > > >> >>>>>>>>>>before allow acls. To give you an example suppose 
>>we
>> > >>have
>> > >> > >
>> > >> > > >> >>>>>>>>>>following acls
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>acl1 -> user1 is allowed to READ from all hosts.
>> > >> > >
>> > >> > > >> >>>>>>>>>>acl2 -> host1 is allowed to READ regardless of 
>>who is
>> > >>the
>> > >> > >
>> > >> > > >>user.
>> > >> > >
>> > >> > > >> >>>>>>>>>>acl3 -> host2 is allowed to READ regardless of 
>>who is
>> > >>the
>> > >> > >
>> > >> > > >>user.
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>acl4 -> user1 is denied to READ from host1.
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>As stated in the KIP we first evaluate DENY so if
>> user1
>> > >> > >
>> > >> > > >> >>>>>>>>>>tries to access from host1 he will be 
>>denied(acl4),
>> > >>even
>> > >> > >
>> > >> > > >> >>>>>>>>>>though both user1 and
>> > >> > >
>> > >> > > >> >>>>>>>>>>host1
>> > >> > >
>> > >> > > >> >>>>>>>>>>has
>> > >> > >
>> > >> > > >> >>>>>>>>>>acl's for allow with wildcards (acl1, acl2).
>> > >> > >
>> > >> > > >> >>>>>>>>>>If user1 tried to READ from host2 , the action 
>>will
>> be
>> > >> > >
>> > >> > > >> >>>>>>>>>>allowed
>> > >> > >
>> > >> > > >> and
>> > >> > >
>> > >> > > >> >>>>>>>>>>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>does
>> > >> > >
>> > >> > > >> >>>>>>>>>>not matter if we match acl3 or acl1 so I don't 
>>think
>> > >>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>evaluation order matters here.
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>"Will people actually use hosts with users?" I 
>>really
>> > >> > > >> >>>>>>>>>>don't
>> > >> > >
>> > >> > > >> >>>>>>>>>>know but given ACl's are part of our Public APIs I
>> > >> > > >> >>>>>>>>>>thought
>> > >> > >
>> > >> > > >> >>>>>>>>>>it is better to try and cover more use cases. If
>> others
>> > >> > >
>> > >> > > >> >>>>>>>>>>think this extra complexity is not
>> > >> > >
>> > >> > > >> worth
>> > >> > >
>> > >> > > >> >>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>value its adding please raise your concerns so we 
>>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>discuss if it should be removed from the acl
>> structure.
>> > >> > >
>> > >> > > >> >>>>>>>>>>Note that even in absence of hosts from ACL users
>> will
>> > >> > >
>> > >> > > >> >>>>>>>>>>still be able to whitelist/blacklist host as long 
>>as
>> we
>> > >> > >
>> > >> > > >> >>>>>>>>>>start supporting principalType = "host", easy to 
>>add
>> > >>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>can be
>> > >> > >
>> > >> > > >> an
>> > >> > >
>> > >> > > >> >>>>>>>>>>incremental improvement. They will however loose 
>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>ability to restrict access to users just from a 
>>set
>> of
>> > >> > hosts.
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>We agreed to offer a CLI to overcome the JSON acl
>> > >>config
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >>
>> > >>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authori
>> > >> > >
>> > >> > > >> >>>>>>>>>>za
>> > >> > >
>> > >> > > >> >>>>>>>>>>ti
>> > >> > >
>> > >> > > >> >>>>>>>>>>on
>> > >> > >
>> > >> > > >> >>>>>>>>>>+I
>> > >> > >
>> > >> > > >> >>>>>>>>>>n
>> > >> > >
>> > >> > > >>
>> > >>>>>>>>>>>>terface#KIP-11-AuthorizationInterface-AclManagement(CLI).
>> > >> > > >> >>>>>>>>>>I
>> > >> > >
>> > >> > > >> >>>>>>>>>>still like Jsons but that probably has something 
>>to
>> do
>> > >> > > >> >>>>>>>>>>with
>> > >> > >
>> > >> > > >> >>>>>>>>>>me being a developer :-).
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>On 4/22/15, 11:38 AM, "Jeff Holoman"
>> > >> > >
>> > >> > > >> 
>>>>>>>>>>>><jholo...@cloudera.com<mailto:jholo...@cloudera.com
>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Parth,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>This is a long thread, so trying to keep up here,
>> > >>sorry
>> > >> > > >> >>>>>>>>>>>if
>> > >> > >
>> > >> > > >> >>>>>>>>>>>this has been covered before. First, great job on
>> the
>> > >> > > >> >>>>>>>>>>>KIP
>> > >> > >
>> > >> > > >> >>>>>>>>>>>proposal and work so far.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Are we sure that we want to tie host level access
>> to a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>given user?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>My
>> > >> > >
>> > >> > > >> >>>>>>>>>>>understanding is that the ACL will be (omitting 
>>some
>> > >> > >
>> > >> > > >> >>>>>>>>>>>fields)
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>user_a, host1, host2, host3 user_b, host1, host2,
>> > >>host3
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>So there would potentially be a lot of 
>>redundancy in
>> > >>the
>> > >> > >
>> > >> > > >> configs.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Does
>> > >> > >
>> > >> > > >> >>>>>>>>>>>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>make sense to have hosts be at the same level as
>> > >> > > >> >>>>>>>>>>>principal
>> > >> > >
>> > >> > > >> >>>>>>>>>>>in
>> > >> > >
>> > >> > > >> the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>hierarchy? This way you could just blanket the
>> > >>allowed /
>> > >> > >
>> > >> > > >> >>>>>>>>>>>denied hosts and only have to worry about the 
>>users.
>> > >>So
>> > >> > > >> >>>>>>>>>>>if
>> > >> > >
>> > >> > > >> >>>>>>>>>>>you follow this, then
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>we can wildcard the user so we can have a 
>>separate
>> > >>list
>> > >> > > >> >>>>>>>>>>>of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>just host-based access. What's the order that the
>> > >>perms
>> > >> > >
>> > >> > > >> >>>>>>>>>>>would be evaluated if a there was more than one
>> match
>> > >>on
>> > >> > > >> >>>>>>>>>>>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>principal ?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Is the thought that there wouldn't usually be 
>>much
>> > >> > > >> >>>>>>>>>>>overlap
>> > >> > >
>> > >> > > >> >>>>>>>>>>>on hosts?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>guess I can imagine a scenario where I want to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>offline/online access to a particular hosts or 
>>set
>> of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>hosts and if there was overlap, I'm doing a 
>>bunch of
>> > >> > > >> >>>>>>>>>>>alter
>> > >> > >
>> > >> > > >> >>>>>>>>>>>commands for just a single host. Maybe this is
>> > >> > >
>> > >> > > >> too
>> > >> > >
>> > >> > > >> >>>>>>>>>>>contrived
>> > >> > >
>> > >> > > >> >>>>>>>>>>>an example?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>I agree that having this level of granularity 
>>gives
>> > >> > >
>> > >> > > >> >>>>>>>>>>>flexibility but I wonder if people will actually 
>>use
>> > >>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>and not just * the hosts for a given user and 
>>create
>> > >> > >
>> > >> > > >> >>>>>>>>>>>separate "global" list as i mentioned above?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>The only other system I know of that ties users 
>>with
>> > >> > > >> >>>>>>>>>>>hosts
>> > >> > >
>> > >> > > >> >>>>>>>>>>>for access is MySql and I don't love that model.
>> > >> > > >> >>>>>>>>>>>Companies
>> > >> > >
>> > >> > > >> >>>>>>>>>>>usually standardize on group authorization 
>>anyway,
>> are
>> > >> > > >> >>>>>>>>>>>we
>> > >> > >
>> > >> > > >> >>>>>>>>>>>complicating that issue with the inclusion of 
>>hosts
>> > >> > >
>> > >> > > >> >>>>>>>>>>>attached to users? Additionally I worry about the
>> debt
>> > >> > > >> >>>>>>>>>>>of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>big JSON configs in the first place, most
>> > >>non-developers
>> > >> > >
>> > >> > > >> >>>>>>>>>>>find them non-intuitive already, so anything to 
>>ease
>> > >> > > >> >>>>>>>>>>>this
>> > >> > >
>> > >> > > >> >>>>>>>>>>>I think would be beneficial.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Jeff
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>On Wed, Apr 22, 2015 at 2:22 PM, Parth 
>>Brahmbhatt <
>> > >> > >
>> > >> > > >> >>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:
>> > >> > > pbrahmbh...@hortonworks.com>> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> Sorry I missed your last questions. I am +0 on
>> > >>adding
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>―host option for  ―list, we could add it for
>> > >>symmetry.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Again if this is only a CLI change it  can be 
>>added
>> > >> > > >> >>>>>>>>>>>>later
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>if you mean adding this in authorizer interface
>> then
>> > >>we
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>should make a decision now.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> Given a choice I would like to actually keep 
>>only
>> > >>one
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>option which is  resource based get (remove even
>> the
>> > >> > > >> >>>>>>>>>>>>get
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>based on principal). I see those  (getAcl for
>> > >>principal
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>or
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>host) as special filtering case which can easily
>> be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>achieved by a third party tool by doing "list 
>>all
>> > >> topics"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>calling
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> getAcls for each topic and applying filtering
>> logic
>> > >>on
>> > >> > >
>> > >> > > >>that.
>> > >> > >
>> > >> > > >> I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>really
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> don't see the need to make those first class
>> > >>citizens
>> > >> > > >> >>>>>>>>>>>> of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the authorizer  interface given these kind of
>> queries
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>will be issued outside
>> > >> > >
>> > >> > > >> of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>broker
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>JVM
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> so they will not benefit from the caching and
>> > >>because
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the storage will be  indexed on resource both 
>>these
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>options even as a first class API will just  
>>scan
>> all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>topic acls and apply filtering logic.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> On 4/22/15, 11:08 AM, "Parth Brahmbhatt"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:
>> > >> > > pbrahmbh...@hortonworks.com>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Please see all the available options here
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Autho
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>ri
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>za
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>ti
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>on
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>+
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >>>nterface#KIP-11-AuthorizationInterface-AclManagement(
>> > >> > > >> >>>>>>>>>>>> >CL
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >) . I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>think
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >covers both hosts and operations and allows to
>> > >> > > >> >>>>>>>>>>>> >specify
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >a list
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>both.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >From: Tom Graves
>> > >> > >
>> > >> > > >> 
>>>>>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com
>> > >> <mailto:
>> > >> > > tgraves...@yahoo.com%3cmailto:tgraves...@yahoo.com>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Reply-To: Tom Graves
>> > >> > >
>> > >> > > >> 
>>>>>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com
>> > >> <mailto:
>> > >> > > tgraves...@yahoo.com%3cmailto:tgraves...@yahoo.com>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Date: Wednesday, April 22, 2015 at 11:02 AM
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >To: Parth Brahmbhatt
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:
>> > >> > >
>> > >> > > >> 
>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >"dev@kafka.apache.org<mailto:
>> dev@kafka.apache.org
>> > >> > > 
>>><mailto:dev@kafka.apache.org%3cmailto:dev@kafka.apache.org%3e>"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> ><dev@kafka.apache.org<mailto:
>> dev@kafka.apache.org
>> > >> > <mailto:
>> > >> > > dev@kafka.apache.org%3cmailto:dev@kafka.apache.org>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Subject: Re: [DISCUSS] KIP-11- Authorization
>> design
>> > >> > > >> >>>>>>>>>>>> >for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >kafka
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>security
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Thanks for the explanations Parth.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >On the configs questions, the way I see it is 
>>its
>> > >> > > >> >>>>>>>>>>>> >more
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >likely
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >accidentally give everyone access, especially
>> since
>> > >> > > >> >>>>>>>>>>>> >you
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >have
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>run
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >separate command to change the acls. If there 
>>was
>> > >> > > >> >>>>>>>>>>>> >some
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >config
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >defaults, a cluster admin could change that 
>>to be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >nobody or
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>certain
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>set
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >of users, then grant others permissions.  This
>> > >>would
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >also
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>remove
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>race
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >between commands.  This is something you can
>> always
>> > >> > > >> >>>>>>>>>>>> >add
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >later
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>though
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>if
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >people request it.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >So in kafka-acl.sh how do I actually tell it 
>>what
>> > >>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>operation
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>is?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >kafka-acl.sh --topic testtopic --add
>> > >>--grandprincipal
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>user:joe,user:kate
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >where does READ, WRITE, etc go?  Can specify 
>>as a
>> > >> > > >> >>>>>>>>>>>> >list
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >so I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>don't
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>have
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >run this a bunch of times for each.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Do you want to have a --host option for 
>>--list so
>> > >> > > >> >>>>>>>>>>>> >that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >admins
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>could
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>see
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >what acls apply to specific host(s)?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Tom
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >On Wednesday, April 22, 2015 11:38 AM, Parth
>> > >> > > >> >>>>>>>>>>>> >Brahmbhatt
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:
>> > >> > >
>> > >> > > >> 
>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >FYI, I have modified the KIP to include group 
>>as
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >resource. In
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>order
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >access "joinGroup" and "commitOFfset" APIs the
>> user
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >will need
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>read
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >permission on topic and WRITE permission on
>> group.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >I plan to open a VOTE thread by noon if there 
>>are
>> > >>no
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >more
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>concerns.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >On 4/22/15, 9:03 AM, "Tom Graves"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>><tgraves...@yahoo.com.INVALID<mailto:
>> > >> > >
>> > >> > > >> 
>>tgraves...@yahoo.com.INVAL<mailto:tgraves...@yahoo.com.INVAL>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>ID
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>Hey everyone,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>Sorry to jump in on the conversation so late.
>> I'm
>> > >> > > >> >>>>>>>>>>>> >>new
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Kafka.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>I'll
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>apologize in advance if you have already 
>>covered
>> > >> > > >> >>>>>>>>>>>> >>some
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>of my
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>questions.  I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>read through the wiki and had some comments 
>>and
>> > >> > >
>> > >> > > >>questions.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>1) public enum Operation needs EDIT changed 
>>to
>> > >>ALTER
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>    Done.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>2) Does the Authorizer class need a setAcls?
>> > >>Rather
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>then
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>just
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>add
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>able to set to explicit list and overwrite 
>>what
>> > >>was
>> > >> > >
>> > >> > > >>there?
>> > >> > >
>> > >> > > >> I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>see
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>kafka-acl.sh lists a removeall so I guess you
>> > >>could
>> > >> > > >> >>>>>>>>>>>> >>do
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>removeall
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>then
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>add.  I also don't see a removeall in the
>> > >>Authorizer
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>class,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>is
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>going
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>to loop through them all to remove each one?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    There is an overloaded version of 
>>removeAcls
>> in
>> > >> > > >> >>>>>>>>>>>> > the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>interface
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >takes
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >in resource as the only input and as 
>>described in
>> > >>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >javadoc
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>acls
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >attached to that resource will be deleted. To
>> cover
>> > >> > > >> >>>>>>>>>>>> >the
>> > >> > >
>> > >> > > >> setAcl
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>use
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>case
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >the caller can first call remove and then add.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>3) Can someone tell me what the use case to 
>>do
>> > >>acls
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>based on
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>hosts?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>I can see some possibilities just wondering 
>>if
>> we
>> > >> > > >> >>>>>>>>>>>> >>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>concrete
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>ones
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>where
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>one user is allowed from one host but not
>> another.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    I am not sure if I understand the question
>> > >>given
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> > the use
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>case
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>you
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >described in your question is what we are 
>>trying
>> to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >cover
>> > >> > >
>> > >> > > >> with
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>use
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >hosts in Acl. There are some additional use 
>>cases
>> > >> > > >> >>>>>>>>>>>> >like
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >"allow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>access
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >any user from host1,host2" but I think 
>>primarily
>> it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >gives the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>admins
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >ability to define acls at a more granular 
>>level.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>4) I'm a bit unclear how the "resource" 
>>works in
>> > >>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Authorizer
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>class.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>From what I see we have 2 resources - topics 
>>and
>> > >> > cluster.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>If I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>want
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>add an acl to allow "joe" to CREATE for the
>> > >>cluster
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>then I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>call
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>addAcls
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>with  Acl("user: joe", ALLOW, Set(*),
>> Set(CREATE))
>> > >> > > >> >>>>>>>>>>>> >>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>"cluster"?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>What
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>if I want to call addAcls for DESCRIBE on a
>> topic?
>> > >> > > >> >>>>>>>>>>>> >>Is
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>resource
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>then
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>"topic" or is it the topic name?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    We now have 3 resources(added group), 
>>please
>> > >>see
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> > the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>updated
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>doc.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>The
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >CREATE acl that you described is correct. For 
>>any
>> > >> > > >> >>>>>>>>>>>> >topic
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>operation
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>you
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >should use topic name as the resource name and
>> for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >group the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>user
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>will
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >provide groupId as resource name.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>5) reassigning partitions is a 
>>CLUSTER_ACTION or
>> > >> > >
>> > >> > > >>superuser?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Its
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>totally clear to me the differences between
>> these.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>what
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>about
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>increasing
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >># of partitions?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    I see this as an alter topic operation so 
>>it
>> is
>> > >> > > >> >>>>>>>>>>>> > at
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> > topic
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>level
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >user must have alter permissions on topic.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>6) groups are mentioned, are we supporting 
>>right
>> > >> > > >> >>>>>>>>>>>> >>away
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>or is
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>follow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>on item? (is there going to be a
>> > >>kafka.supergroups)
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    I think it can be a separate jira just for
>> > >> > > >> >>>>>>>>>>>> > braking
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> > down
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>code
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >review
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >in smaller chunk. We will support it in first
>> > >>version
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >but I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>think
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>if
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>we
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >can not do it for any reason that should not
>> block
>> > >>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >release
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>with
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >other authZ work. We made deliberate design
>> choices
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >(like
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>introducing
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >principalType in KafkaPrinciapl) to allow
>> > >>supporting
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >groups
>> > >> > >
>> > >> > > >> as
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>an
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >incremental change.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>7) Are there config options for setting acls
>> when
>> > >>I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>create
>> > >> > >
>> > >> > > >> my
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>topic?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Or
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>do I have to create my topic and then run the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>kafka-acl.sh
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>script
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>set
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>them?  Although its very small, there would 
>>be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>possible race
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>there
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>someone could start producing to topic before
>> acls
>> > >> > > >> >>>>>>>>>>>> >>are
>> > >> > >
>> > >> > > >>set.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    We discussed this yesterday and we agreed 
>>to
>> go
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> > with
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>kafka-acl.sh.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Yes
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >there is a very very small window of
>> vulnerability
>> > >> > > >> >>>>>>>>>>>> >but
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >I
>> > >> > >
>> > >> > > >> think
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>really
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >does not warrant to change the decision in 
>>this
>> > >>case.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>8) are there configs for cluster level acl
>> > >>defaults?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>Or
>> > >> > >
>> > >> > > >> does
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>default
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>to superusers on bringing up new cluster and 
>>you
>> > >> > > >> >>>>>>>>>>>> >>have
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>modify
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>with
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>cli.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>thanks,Tom
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >    No defaults, the default is superusers 
>>will
>> > >>have
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> > full
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>access.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>don't
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >think making assumptions about ones security
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >requirement
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>should
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>our
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >burden.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>    On Tuesday, April 21, 2015 7:10 PM, Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >> Brahmbhatt
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:
>> > >> > >
>> > >> > > >> 
>>pbrahmbh...@hortonworks.co<mailto:pbrahmbh...@hortonworks.co>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>>m>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >> I have added the notes to KIP-11 Open 
>>question
>> > >> > sections.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>On 4/21/15, 4:49 PM, "Gwen Shapira"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com
>> > >> > > <mailto:gshap...@cloudera.com%3cmailto:gshap...@cloudera.com>>>
>> > >> > >
>> > >> > > >> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>Adding my notes from today's call to the
>> thread:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>** Deny or Allow all by default? We will 
>>add a
>> > >> > >
>> > >> > > >> configuration
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>control this. The configuration will 
>>default to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>"allow" for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>backward
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>compatibility. Security admins can set it to
>> > >>"deny"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>** Storing ACLs for default authorizers: 
>>We'll
>> > >> > > >> >>>>>>>>>>>> >>>store
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>them
>> > >> > >
>> > >> > > >> in
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>ZK.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>We'll
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>support pointing the authorizer to any ZK.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>The use of ZK will be internal to the 
>>default
>> > >> > >
>> > >> > > >>authorizer.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>Authorizer
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>reads ACLs from cache every hour. We 
>>proposed
>> > >> > > >> >>>>>>>>>>>> >>>having
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>mechanism
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>(possibly via new ZK node) to tell broker to
>> > >> > > >> >>>>>>>>>>>> >>>refresh
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>cache
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>immediately.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>** Support deny as permission type - we 
>>agreed
>> to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>keep
>> > >> > >
>> > >> > > >> this.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>** Mapping operations to API: We may need to
>> add
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>Group as a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>resource,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>with JoinGroup and OffsetCommit require
>> privilege
>> > >> > > >> >>>>>>>>>>>> >>>on
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>consumer
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>group.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>This can be something we pass now and
>> authorizers
>> > >> > > >> >>>>>>>>>>>> >>>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>support
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>in
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>future. - Jay will write specifics to the
>> mailing
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>list
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>discussion.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>><jay.kr...@gmail.com<mailto:
>> jay.kr...@gmail.com
>> > >> > <mailto:
>> > >> > > jay.kr...@gmail.com%3cmailto:jay.kr...@gmail.com>>> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> Following up on the KIP discussion. Two
>> options
>> > >> > > >> >>>>>>>>>>>> >>>> for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>authorizing
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>consumers
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> to read topic "t" as part of group "g":
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> 1. READ permission on resource /topic/t  
>>2.
>> > >>READ
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>permission on resource /topic/t AND WRITE
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>permission
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>on
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>/group/g
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> The advantage of (1) is that it is 
>>simpler.
>> The
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>disadvantage
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>is
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>any
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> member of any group that reads from t can
>> > >>commit
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>offsets
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>as
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>any
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>other
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> member of a different group. This doesn't
>> > >>effect
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> data
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>security
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>(who
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> access what) but it is a bit of a 
>>management
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>issue--a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>malicious
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>person
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> cause data loss or duplicates for another
>> > >> > > >> >>>>>>>>>>>> >>>> consumer
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>by
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>committing
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>offset.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> I think I favor (2) but it's worth it to
>> think
>> > >>it
>> > >> > >
>> > >> > > >> through.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> -Jay
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>> On Tue, Apr 21, 2015 at 2:43 PM, Parth
>> > >>Brahmbhatt
>> > >> > > >> >>>>>>>>>>>> >>>> <
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>
>> > >> > >
>> > >> > > >>
>> > >>>>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:pbrahmbhatt@hortonwo
>> > >> > > >> >>>>>>>>>>>>rk
>> > >> > > <mailto:pbrahmbh...@hortonworks.com
>> %3cmailto:pbrahmbhatt@hortonwork
>> > >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>s
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>.com
>> > >> > >
>> > >> > > >> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> Hey Jun,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> Yes and we support wild cards for all acl
>> > >> > > >> >>>>>>>>>>>> >>>>> entities
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>principal,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>hosts
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> operation.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> On 4/21/15, 9:06 AM, "Jun Rao"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >>>>>>><j...@confluent.io<mailto:j...@confluent.io<mailto:
>> > >> > > j...@confluent.io%3cmailto:j...@confluent.io>>> wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >Harsha, Parth,
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >Thanks for the clarification. This makes
>> > >>sense.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >Perhaps
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>we
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>clarify the
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >meaning of those rules in the wiki.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >Related to this, it seems that we need 
>>to
>> > >> > > >> >>>>>>>>>>>> >>>>> >support
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>wildcard
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>in
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>cli/request
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >protocol for topics?
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >Jun
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >On Mon, Apr 20, 2015 at 9:07 PM, Parth
>> > >> > > >> >>>>>>>>>>>> >>>>> >Brahmbhatt
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> ><
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto<mailto:
>> > >> > > pbrahmbh...@hortonworks.com%3cmailto>:
>> > >> > >
>> > >> > > >> 
>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> The iptables on unix supports the DENY
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> operator, not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>it
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>should
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> matter. The deny operator can also be
>> used
>> > >>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> specify
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>³allow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>user1
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>READ
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> from topic1 from all hosts but
>> > >>host1,host2².
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>Again we
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>could
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>add a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>host
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> group semantic and extra complexity
>> around
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> that, not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>sure
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>if
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>its
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>worth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>it.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> In addition with DENY operator you are
>> now
>> > >> > > >> >>>>>>>>>>>> >>>>> >> not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>forced
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>create a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>special
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> group just to support the 
>>authorization
>> use
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>case. I
>> > >> > >
>> > >> > > >> am
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>convinced
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> the operator it self is really all 
>>that
>> > >> > confusing.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>There
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>are 3
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>practical
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> use cases:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> - Resource with no acl what so ever ->
>> > >>allow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> access
>> > >> > >
>> > >> > > >> to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>everyone (
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>just
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>for
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> backward compatibility, I would much
>> rather
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>fail
>> > >> > >
>> > >> > > >> close
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>force
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>users
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> explicitly grant acls that allows 
>>access
>> to
>> > >> > > >> >>>>>>>>>>>> >>>>> >> all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>users.)
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> - Resource with some acl attached -> 
>>only
>> > >> > > >> >>>>>>>>>>>> >>>>> >> users
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>have
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>matching
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>allow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> acl are allowed (i.e. ³allow READ 
>>access
>> to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>topic1 to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>user1
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>from
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> hosts², only user1 has READ access 
>>and no
>> > >> > > >> >>>>>>>>>>>> >>>>> >> other
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> user
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>has
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>access of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>any
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> kind)
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> - Resource with some allow and some 
>>deny
>> > >>acl
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> attached
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>->
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>users
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>are
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>allowed
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> to perform operation only when they
>> satisfy
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>allow acl
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>do
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>have
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> conflicting deny acl. Users that have 
>>no
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> acl(allow or
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>deny)
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>will
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>still
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> have any access. (i.e. ³allow READ 
>>access
>> > >>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>topic1
>> > >> > >
>> > >> > > >> to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>user1
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>from
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> hosts except host1 and host², only 
>>user1
>> > >>has
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> access
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>but
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>not
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>from
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>host1
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>an
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> host2)
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> I think we need to make a decision on
>> deny
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> primarily
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>because
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>with
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> introduction of acl management API, 
>>Acl
>> is
>> > >> > > >> >>>>>>>>>>>> >>>>> >> now
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> a
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>public
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>class
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>that
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>will
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> used by Ranger/Santry and other
>> > >>authroization
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>providers.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>In
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>Current
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>design
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> the acl has a permissionType enum 
>>field
>> > >>with
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>possible
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>values
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>Allow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>and
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> Deny. If we chose to remove deny we 
>>can
>> > >> > > >> >>>>>>>>>>>> >>>>> >> assume
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>all
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>acls
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>to
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>be
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>of
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>allow
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> type and remove the permissionType 
>>field
>> > >> > >
>> > >> > > >>completely.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> Thanks
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> Parth
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> On 4/20/15, 6:12 PM, "Gwen Shapira"
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >>>>>>><gshap...@cloudera.com<mailto:gshapira@cloudera.c
>> > >> > > >> >>>>>>>>>>>> >>>>>om
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>wrote:
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> >I think thats how its done in pretty
>> much
>> > >> > > >> >>>>>>>>>>>> >>>>> >> >any
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> >system
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>I
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>can
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>think
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>of.
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>> >
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>>>--
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Jeff Holoman
>> > >> > >
>> > >> > > >> >>>>>>>>>>>Systems Engineer
>> > >> > >
>> > >> > > >> >>>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>>>
>> > >> > >
>> > >> > > >> >>>>>>>
>> > >> > >
>> > >> > > >> >>>>>
>> > >> > >
>> > >> > > >> >>>
>> > >> > >
>> > >> > > >> >
>> > >> > >
>> > >> > > >>
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> >
>>

Re: [VOTE] KIP-11- Authorization design for kafka security

Reply via email to