Hmm, I thought the semantics is that if you only have rule "deny user2", it
means that everyone except user2 has access?
Thanks,
Jun
On Mon, Apr 20, 2015 at 3:25 PM, Parth Brahmbhatt <
pbrahmbh...@hortonworks.com> wrote:
> user3 does not have access and removing the deny rule does not grant him
> or user2 access. user2 even without the deny rule will not have access.
>
> Thanks
> Parth
>
> On 4/20/15, 12:03 PM, "Jun Rao" <j...@confluent.io> wrote:
>
> >Just a followup question. Suppose there are two rules. Rule1 allows user1
> >and rule2 denies user2. Does user3 have access? If not, does removing
> >rule1
> >enable user3 access?
> >
> >Thanks,
> >
> >Jun
> >
> >On Mon, Apr 20, 2015 at 1:34 PM, Parth Brahmbhatt <
> >pbrahmbh...@hortonworks.com> wrote:
> >
> >>
> >> Hi Joel,
> >>
> >> Thanks for the review and I plan to update the KIP today with all the
> >> updated info. My comments in line below.
> >>
> >> Thanks
> >> Parth
> >>
> >>
> >> On 4/20/15, 10:07 AM, "Joel Koshy" <jjkosh...@gmail.com<mailto:
> >> jjkosh...@gmail.com>> wrote:
> >>
> >> Hi Parth,
> >>
> >> Nice work on this KIP. I did another read through and had a few more
> >> comments (with edits after I went through the thread). Many of these
> >> comments were brought up by others as well, so it appears that the KIP
> >> would benefit from an update at this point to incorporate comments
> >> from the thread and last hangout.
> >>
> >> - The operation enum is mostly self-explanatory, but it would help
> >> (for the sake of clarity and completeness if nothing else) to
> >> document exactly what each of the enums are. E.g., I think this came
> >> up in our hangout - SEND_CONTROL_MESSAGE is unclear and I don't
> >> remember what was said about it. <Edit>: After going through the
> >> thread it seems the conclusion was to categorize operations. E.g.,
> >> WRITE could apply to multiple requests. Again, this is unclear, so
> >> if it would be great if you could update the KIP to clarify what you
> >> intend.
> >>
> >> Will add to document. SEND_CONTROL_MESSAGE Probably a very bad name but
> >> these are intra borker API calls like controller notifying other
> >>brokers to
> >> update metadata or heartbeats. Any better naming suggestions?
> >>
> >> - When you update the KIP to categorize the requests it would also
> >> help to have a column for what the resource is for each.
> >>
> >> Will add to the KIP.
> >>
> >> - FWIW I prefer a 1-1 mapping between requests and operations. I think
> >> categorizing requests into these can be confusing because:
> >> - The resource being protected for different requests will be
> >> different. We are mostly thinking about topics (read/write) but
> >> there are requests for which topic is not the right resource.
> >> E.g., for topic creation, the resource as you suggested would be
> >> something global/common such as “cluster”. For
> >> OffsetCommit/FetchRequest, the resource may be the consumer group,
> >> or maybe a tuple of <consumer group, topic>. So this can be
> >> confusing - i.e., different resources and request types in the
> >> same category. It may be simpler and clearer to just have a 1-1
> >> mapping between the operation enum and requests.
> >>
> >> I only see 2 resource categories right now cluster and topic. I don’t
> >> really care one way or another so we can probably make a quick decision
> >>in
> >> tomorrow’s meeting to either to 1-1 mapping or have categorization?
> >>
> >> - Some requests that are intuitively READ have WRITE side-effects.
> >> E.g., (currently) TopicMetadataRequest with auto-create, although
> >> that will eventually go away. ConsumerMetadataRequest still
> >> auto-creates the offsets topic. Likewise, ADMIN-type requests may
> >> be interpreted as having side-effects (depending on who you ask).
> >>
> >> Yes and what I am doing right now is checking authorization for all
> >> possible actions i.e. for auto-create it checks if the config has it
> >> enabled and if yes, check for read + create authorization. Its not very
> >> meaningful right now as there is no CREATE authorization but I think
> >>this
> >> is implementation detail, we need to ensure we call authorize with all
> >> possible operations from KafkaAPI.
> >> - <quote>When an ACL is missing - fail open</quote>. What does missing
> >> mean? i.e., no explicit ACL for a principal? I'm confused by this
> >> especially in relation to the precedence of DENY over ALLOW. So per
> >> the description:
> >> - If no ACLs exist for topic A then ALLOW all operations on it by
> >> anyone.
> >> - If I now add an ACL for a certain principal P to ALLOW (say) WRITE
> >> to the topic then either:
> >> - This has the effect of DENYing WRITE to all other principals
> >> - Or, this ACL serves no purpose
> >> - If the effect is to DENY WRITE to all other principals, what about
> >> READ. Do all principals (including P) have READ permissions to
> >> topic A?
> >> - In other words, it seems for a specific ACL to be meaningful then
> >> fail close is necessary for an absent ACL.
> >> - <edit>After through the thread: it appears that the DENY override
> >> only applies to the given principal. i.e., in the above case it
> >> appears that the other principals will in fact be granted access.
> >> Then this makes the ACL that was added pointless right?
> >>
> >> The rule I was going with is
> >> - If there is no ACL I.e. This might be a topic that was created in non
> >> secure mode or was created before we supported ACLs. We assume you do
> >>not
> >> want authorization and let all requests go through.
> >> - once you add any ACL, we assume you want authorization on the topic
> >>and
> >> all the general authorization rules now start to apply, I.e we fail
> >>close
> >> if we don’t find an ACL that allows access or if we find an ACL that
> >>denies
> >> access. It does not matter if you added a READACL or WRITEACL or
> >>ALLOWACL
> >> or DENY ACL. If you add any ACL, now every user gets checked against
> >>that
> >> and if it does not satisfy the ACL, request fails. I.e. If you add an
> >>ACL
> >> “Allow write to topic-1 form user1 from all hosts” , user-1 has write
> >> access from all hosts and no other user has any access(except for
> >> superusers who have all the access).
> >> - Deny ACLS are suppose to be used to restrict access authorized by some
> >> allow ACL, they are not suppose to be required. Implicitly anyone who
> >>does
> >> not have an allow acl, gets denied. The Deny ACLs are only added to give
> >> more control to administrators who wants more granular control with
> >>lesser
> >> config. The scenario described in mailing list was “Allow user X access
> >> from all hosts but Host1,Host2”. in absence of DENY operator you will
> >>have
> >> to exhaustively list all possible hosts in your ACL which is what we are
> >> trying to avoid.
> >>
> >> - On ZK ACLs: I think ZK will be closed to everyone except Kafka
> >> brokers. This is a dependency on KIP-4 though. i.e., eventually all
> >> clients should talk to brokers only via RPC.
> >>
> >> Yes.
> >>
> >> - Topic owner: list vs single entry - both have issues off the bat
> >> (although list is more intuitive at least to me), but perhaps you
> >> could write up some example workflows to clarify the current
> >> proposal. I was thinking that anyone in the owner list should be
> >> considered a super-user of the topic and can grant/revoke
> >> permissions. They should also be allowed to add other principals as
> >> owners. Even with this it is unclear who should be allowed to remove
> >> owners.
> >>
> >> As you pointed out in the last KIP meeting owners/creators have use out
> >> side of security context (plain simple auditing). I don’t think the
> >> authorizer work depends on this, it was my bad to even mention it in
> >>first
> >> place. I think we can have this discussion outside of
> >>authorizer/security
> >> context and once we have a way to get topic owners the default
> >>Authorizer
> >> can start using it. It makes sense to treat all owners as super users
> >>and I
> >> think it is safe to assume superusers can also modify ownership but I
> >>think
> >> this should not be treated as blocking work for authorization.
> >>
> >> - What is the effect of deleting a topic - should all associated ACLs
> >> be deleted as well?
> >> They should be and with acls being stored as part of TopicConfig this
> >>was
> >> taken care of automatically. With the new ACL management API the users
> >>will
> >> have to call remove ACLs explicitly to perform the cleanup. If everyone
> >> thinks this should be automated , with the new APIs we will need a
> >>hook(or
> >> poll) to be notified when a topic is deleted to perform cleanup.
> >> - TopicConfigCache to store topic-ACLs. As mentioned above, not all
> >> requests will be tied to topics. We may want to have an entirely
> >> separate ZK directory for ACLs. We have a similar issue with quotas.
> >> This ties in with dynamic config management. We can certainly
> >> leverage the dynamic config management part of topic configs but I
> >> think we need to have a story for non-topic resources.
> >>
> >> In the first proposal I was going with a topic-Acl and cluster-Acl where
> >> cluster-Acls were json acl local files on all brokers. With the new ACL
> >> management APIs we are planning to have /kafka-acl node under which all
> >> acls will be stored in /kakfa-acls/resource-name -> {acl json data}.
> >> Cluster acls will just have resource name kafka-cluster.
> >>
> >> Thanks,
> >>
> >> Joel
> >>
> >> On Thu, Apr 16, 2015 at 12:15:37AM +0000, Parth Brahmbhatt wrote:
> >> Kafka currently stores logConfig overrides specified during topic
> >>creation
> >> in zookeeper, its just an instance of java.util.Properties converted to
> >> json. I am proposing in addition to that we store acls and owner as well
> >> as part of same Properties map.
> >> There is some infrastructure around reading this config, converting it
> >> back to Properties map and most importantly propagating any changes
> >> efficiently which we will be able to leverage. As this infrastructure is
> >> common to the cluster the reading (not interpreting) of config happens
> >> outside of any authorization code.
> >> If the TopicConfigCache just kept the json representation and left it to
> >> authorizer to parse it, the authorizer will have to either parse the
> >>json
> >> for each request(not acceptable) or it will have to keep one more layer
> >>of
> >> parsed ACL instance cache. Assuming authorizer will keep an additional
> >> caching layer we will now have to implement some way to invalidate the
> >> cache which means the TopicConfigCache will have to be an observable
> >>which
> >> the Authorizer observes and invalidates its cache entries when
> >> topicConfigCache gets updated. Seemed like unnecessary complexity with
> >>not
> >> lot to gain so I went with TopicConfigCache interpreting the json and
> >> caching a higher level modeled object.
> >> In summary, the interpretation is done for both optimization and
> >> simplicity. If you think it is important to allow custom ACL format
> >> support we can add one more pluggable config(acl.parser) and
> >> interface(AclParser) or it could just be another method in Authorizer.
> >> One thing to note the current ACL json is versioned so it is easy to
> >>make
> >> changes to it however it won’t be possible to support custom ACL formats
> >> with the current design.
> >> Thanks
> >> Parth
> >> On 4/15/15, 4:29 PM, "Michael Herstine" <mherst...@linkedin.com.INVALID
> >> <mailto:mherst...@linkedin.com.INVALID>>
> >> wrote:
> >> >Hi Parth,
> >> >
> >> >I’m a little confused: why would Kafka need to interpret the JSON?
> >>IIRC
> >> >KIP-11 even says that the TopicConfigData will just store the JSON. I’m
> >> >not really making a design recommendation here, just trying to
> >>understand
> >> >what you’re proposing.
> >> >
> >> >On 4/15/15, 11:20 AM, "Parth Brahmbhatt" <pbrahmbh...@hortonworks.com
> >> <mailto:pbrahmbh...@hortonworks.com>>
> >> >wrote:
> >> >
> >> >>Hi Michael,
> >> >>
> >> >>There is code in kafka codebase that reads and interprets the topic
> >> >>config JSON which has acls, owner and logconfig properties. There are
> >>3
> >> >>use cases that we are supporting with current proposal:
> >> >>
> >> >> * You use out of box simpleAcl authorizer which is tied to the acl
> >> >>stored in topic config and the format is locked down.
> >> >> * You have a custom authorizer and a custom ACL store.
> >>Ranger/Argus
> >> >>falls under this as they have their own acl store and ui that users
> >>use
> >> >>to configure acls on the cluster and cluster resources like topic.
> >>It is
> >> >>upto the custom authorizer to leverage the kafka acl configs or
> >> >>completely ignore them as they have set a user expectation that only
> >>acls
> >> >>configured via their ui/system will be effective.
> >> >> * You have a custom authorizer but no custom Acl store. You are
> >> >>completely tied to Acl structure that we have provided in out of box
> >> >>implementation.
> >> >>
> >> >>Thanks
> >> >>Parth
> >> >>
> >> >>On 4/15/15, 10:31 AM, "Michael Herstine"
> >> >><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID
> >> ><mailto:mherst...@linkedin.com.INVALID>>
> >> >>wrote:
> >> >>
> >> >>Hi Parth,
> >> >>
> >> >>One question that occurred to me at the end of today’s hangout: how
> >>tied
> >> >>are we to a particular ACL representation under your proposal? I know
> >> >>that
> >> >>TopicConfigCache will just contain JSON— if a particular site decides
> >> >>they
> >> >>want to represent their ACLs differently, and swap out the authorizer
> >> >>implementation, will that work? I guess what I’m asking is whether
> >> >>there’s any code in the Kafka codebase that will interpret that JSON,
> >>or
> >> >>does that logic live exclusively in the authorizer?
> >> >>
> >> >>On 4/14/15, 10:56 PM, "Don Bosco Durai"
> >> >><bo...@apache.org<mailto:bo...@apache.org><mailto:bo...@apache.org>>
> >> wrote:
> >> >>
> >> >>I also feel, having just IP would be more appropriate. Host lookup
> >>will
> >> >>unnecessary slow things down and would be insecure as you pointed out.
> >> >>
> >> >>With IP, it will be also able to setup policies (in future if needed)
> >> >>with
> >> >>ranges or netmasks and it would be more scalable.
> >> >>
> >> >>Bosco
> >> >>
> >> >>
> >> >>On 4/14/15, 1:40 PM, "Michael Herstine"
> >> >><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID
> >> ><mailto:mherst...@linkedin.com.INVALID>>
> >> >>wrote:
> >> >>
> >> >>Hi Parth,
> >> >>
> >> >>Sorry to chime in so late, but I’ve got a minor question on the KIP.
> >> >>
> >> >>Several methods take a parameter named “host” of type String. Is that
> >> >>intended to be a hostname, or an IP address? If the former, I’m
> >>curious
> >> >>as
> >> >>to how that’s found (in my experience, when accepting an incoming
> >>socket
> >> >>connection, you only know the IP address, and there isn’t a way to map
> >> >>that to a hostname without a round trip to a DNS server, which is
> >> >>insecure
> >> >>anyway).
> >> >>
> >> >>
> >> >>On 3/25/15, 1:07 PM, "Parth Brahmbhatt"
> >>
> >>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
> ><mailto
> >>>>:
> >> pbrahmbh...@hortonworks.com>>
> >> >>wrote:
> >> >>
> >> >>Hi all,
> >> >>
> >> >>I have modified the KIP to reflect the recent change request from the
> >> >>reviewers. I have been working on the code and I have the server side
> >> >>code
> >> >>for authorization ready. I am now modifying the command line
> >>utilities.
> >> >>I
> >> >>would really appreciate if some of the committers can spend sometime
> >>to
> >> >>review the KIP so we can make progress on this.
> >> >>
> >> >>Thanks
> >> >>Parth
> >> >>
> >> >>On 3/18/15, 2:20 PM, "Michael Herstine"
> >> >><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID
> >> ><mailto:mherst...@linkedin.com.INVALID>>
> >> >>wrote:
> >> >>
> >> >>Hi Parth,
> >> >>
> >> >>Thanks! A few questions:
> >> >>
> >> >>1. Do you want to permit rules in your ACLs that DENY access as well
> >>as
> >> >>ALLOW? This can be handy setting up rules that have exceptions. E.g.
> >> >>“Allow principal P to READ resource R from all hosts” with “Deny
> >> >>principal
> >> >>P READ access to resource R from host H1” in combination would allow P
> >> >>to
> >> >>READ R from all hosts *except* H1.
> >> >>
> >> >>2. When a topic is newly created, will there be an ACL created for it?
> >> >>If
> >> >>not, would that not deny subsequent access to it?
> >> >>
> >> >>(nit) Maybe use Principal instead of String to represent principals?
> >> >>
> >> >>
> >> >>On 3/9/15, 11:48 AM, "Don Bosco Durai"
> >> >><bo...@apache.org<mailto:bo...@apache.org><mailto:bo...@apache.org>>
> >> wrote:
> >> >>
> >> >>Parth
> >> >>
> >> >>Overall it is looking good. Couple of questionsŠ
> >> >>
> >> >>- Can you give an example how the policies will look like in the
> >> >>default
> >> >>implementation?
> >> >>- In the operations, can we support ³CONNECT² also? This can be used
> >> >>during Session connection
> >> >>- Regarding access control for ³Topic Creation², since we can¹t do it
> >> >>on
> >> >>the server side, can we de-scope it for? And plan it as a future
> >> >>feature
> >> >>request?
> >> >>
> >> >>Thanks
> >> >>
> >> >>Bosco
> >> >>
> >> >>
> >> >>On 3/6/15, 8:10 AM, "Harsha" <ka...@harsha.io<mailto:ka...@harsha.io
> >> ><mailto:ka...@harsha.io>>
> >> >>wrote:
> >> >>
> >> >>Hi Parth,
> >> >> Thanks for putting this together. Overall it looks good
> >> >>to
> >> >> me. Although AdminUtils is a concern KIP-4 can probably
> >> >>fix
> >> >> that part.
> >> >>Thanks,
> >> >>Harsha
> >> >>
> >> >>On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote:
> >> >>Forgot to add links to wiki and jira.
> >> >>Link to wiki:
> >> >>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza
> >> >>t
> >> >>i
> >> >>o
> >> >>n
> >> >>+
> >> >>Interface
> >> >>Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688
> >> >>Thanks
> >> >>Parth
> >> >>From: Parth Brahmbhatt
> >>
> >>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
> ><mailto
> >>>>:
> >> pbrahmbh...@hortonworks.com><mailto:p
> >> >>b
> >> >>rahmbh...@hortonworks.com<mailto:rahmbh...@hortonworks.com>>>
> >> >>Date: Thursday, March 5, 2015 at 10:33 AM
> >> >>To:
> >> >>"dev@kafka.apache.org<mailto:dev@kafka.apache.org><mailto:
> >> dev@kafka.apache.org><mailto:dev@kafka.apach
> >> >>e
> >> >>.org>"
> >> >><dev@kafka.apache.org<mailto:dev@kafka.apache.org><mailto:
> >> dev@kafka.apache.org><mailto:dev@kafka.apach
> >> >>e
> >> >>.org>>
> >> >>Subject: [DISCUSS] KIP-11- Authorization design for kafka security
> >> >>Hi,
> >> >>KIP-11 is open for discussion , I have updated the wiki with the
> >> >>design
> >> >>and open questions.
> >> >>Thanks
> >> >>Parth
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >>
> >>
> >>
>
>
