@Balaji,
I know how much you love long emails. Here goes! :)
You wrote:
> Let us take the use case we discussed earlier. The group "interns" got
> denied,but we want to exclude one user ("Scott") from that, I am suggesting
> to have the particular user ("Scott") included in the Allow section of policy
> and group "interns" included in Deny section of the same policy. Ranger
> should evaluate the both Allow and Deny in the same policy and deduce that
> user "Scott" can access the resource while anyone else in group "interns" is
> denied to access the resource. The policy creation in the UI is much simpler
> than having to specify "Scott" as an exception to Deny and then specifying
> "Scott" for Allow.
Here’s why what you say can’t be done without also imposing a “hidden/implicit
rules”.
Our use case is as follows:
Finance department is very large. Some Interns are in Finance. Only people in
Finance have access to Budget forecasts. Interns are not given access to
Budget. Scott is a finance intern who is doing a project for CFO and hence
needs access to the budget data.
Madhan’s model proposes to address this with the following policy:
* Allow – Finance
* Deny – Intern
* Deny exception – Scott
Which would, as the use case requires, allow access to everyone in Finance
unless they are an intern, except to Scott. Please note that policy’s
description alone is sufficient for anyone to know what it would do.
Now let us consider the policy that you have suggested:
* Allow – Finance, Scott
* Deny – Interns
For the above to honor our use case we would also need to apply the following
additional rule:
* Allow access if user has been explicitly been given access even if he has
been denied. (This BTW was what Abhay’s had suggested.)
Unless we have the above rule we can’t disambiguate what to do about Scott who
is part of both Finance and Interns — one of which has been given access and
another has been denied. One must have a way to break the tie in the form of a
rule. @Balaji which do you feel is better?
* Your combination of (simpler policy + simple rule) or
* Madhan’s complicated policy without any additional rules
Perhaps the decision is not obvious at this stage. Now, consider the following
use case which is only a slight modification to above.
New use case:
Finance department is very large. Some Interns are in Finance. Everyone from
Finance has access to Budget forecast except its interns. CEO has asked every
department to allocate people to his favorite new project - Compliance.
Finance has asked some of its interns to be part of Compliance project and
wants to give then access to Budget forecast.
Madhan’s Model can address this as follows:
* Allow – Finance
* Deny – Intern
* Deny exception: Compliance
Which again is straight forward to interpret and verify.
You model would have to be as follows:
* Allow – Finance, Compliance
* Deny – Intern
How do we interpret this policy given that some interns are in Finance, some of
those working on Compliance project are in Finance? We would need an
additional rule as follows:
* If a user is explicitly allowed access because of a greater number groups
(Finance and Compliance) than he is denied access (Intern) then allow wins and
he should be allowed access.
So now we have two rules.
The point is this: Domain we want to model is complex. And we have two choices.
* Keep complexity in the structure of your model itself. This makes it
easy to reason about it without the need of an additional codec (set of rules).
In computer science this is often referred to as a Declarative specification,
i.e. you expend your energies only in stating your intent. You are NOT
concerned about how something works. This is Madhan’s style — naturally as
programmers have a bias towards getting to declarative specification.
* Keep the model simple, move complexity to a set of rules that must be
consulted in a known order to get at the result. This is often called as an
Imperative specification. To understand and reason about something it is not
enough to just define your data but you also have to know the rules and know
how they are applied. This has advantage of course. The rules can be easily
re-ordered to yield a system that does something entirely different.
Disadvantages are obvious: as the number of rules grow system becomes hard to
reason about and hence brittle.
I can tell you from years of computer science research the move away from an
imperative style is towards declarative style. We should think long and hard
before we rule in favor of an imperative specification in this day and age.
Thanks
- Alok
From: Don Bosco Durai <[email protected]<mailto:[email protected]>>
Reply-To:
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Friday, October 16, 2015 at 2:10 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
> I agree that we need a UI setting to enable/disable deny and exceptions. For
> general majority, they should not even see any thing related to deny or
> exceptions. Can we agree on this as a needed feature for Ranger?
I am okay with this. But we have to document and advertise this so that users
know that this feature exists and they can enable it.
Bosco
From: Balaji Ganesan
Reply-To:
<[email protected]<mailto:[email protected]>>
Date: Friday, October 16, 2015 at 1:47 PM
To: <[email protected]<mailto:[email protected]>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
<<I am not sure how tag based policy will simplify. I feel, “Deny” is advanced
feature and I would expect the Admin to be smart enough to know what he/she is
doing. Maybe in the future we can give more tools to help the Admins.>>
We all know from experience that expecting Admin to be smart enough does not
always hold true. The premise of doing a UI, and not a command line tool, is
the admins do not have expertise or training to construct access policies for
Hadoop easily. That is big differentiator for Ranger, which is the ease of use
and centralized policy management.
<<Based on the Madhan’s examples, yes it is needed. Without it, there are few
use cases we can’t meet. We can use a convoluted to meet these requirement, but
I feel, “Exceptions/Exclude” might be a simpler option.>>
If there are only few customers who would need this, the deny or allow
exception functionality need not be exposed for the majority. If there is
option available, users will use it even if they could have met their use case
using simpler methods.
<<I agree with this. We can make it optional and have an UI setting to enable
it. Generally, users won’t know this feature exists. I would put a band at the
both of the policy edit page and put a link with “Deny feature is disabled.
Click here to enable it”.>>
I agree that we need a UI setting to enable/disable deny and exceptions. For
general majority, they should not even see any thing related to deny or
exceptions. Can we agree on this as a needed feature for Ranger?
On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai
<[email protected]<mailto:[email protected]>> wrote:
>That is the concern here. How can we expect Administrators to ensure they are
>keeping policies in sync between high level and lower level resources? The use
>case is Scott and finance group should get access while everyone else in
>intern group should. The intern group is part of the finance group. We can
>expect the administrators to achieve this by creating a higher level tag based
>policy and a resource based policy.
I am not sure how tag based policy will simplify. I feel, “Deny” is advanced
feature and I would expect the Admin to be smart enough to know what he/she is
doing. Maybe in the future we can give more tools to help the Admins.
>Not sure if we are discussing the same thing here. My question was around the
>need to have exceptions for deny and allow. If it indeed going to be
>complicated, then should we not such areas which add more complication.
Based on the Madhan’s examples, yes it is needed. Without it, there are few use
cases we can’t meet. We can use a convoluted to meet these requirement, but I
feel, “Exceptions/Exclude” might be a simpler option.
>I think we should provide both deny and exceptions policies as optional. The
>exceptions should be turned off by default and should not appear in the UI.
>Users who really need exceptions can turn it on.
I agree with this. We can make it optional and have an UI setting to enable it.
Generally, users won’t know this feature exists. I would put a band at the both
of the policy edit page and put a link with “Deny feature is disabled. Click
here to enable it”.
Thanks
Bosco
From: Balaji Ganesan
Reply-To:
<[email protected]<mailto:[email protected]>>
Date: Thursday, October 15, 2015 at 10:54 PM
To: <[email protected]<mailto:[email protected]>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
<<I think, Madhan’s use case is not exactly the same. In his case, he wants to
“Deny” all interns at a high level, but not necessary want to Allow (globally)
“interns” in this policy (which could be bad in some cases). Scott’s
permissions to any sub resources might be managed by other policies.>>
That is the concern here. How can we expect Administrators to ensure they are
keeping policies in sync between high level and lower level resources? The use
case is Scott and finance group should get access while everyone else in intern
group should. The intern group is part of the finance group. We can expect the
administrators to achieve this by creating a higher level tag based policy and
a resource based policy.
<<When we decided to support “Deny”, we know it would add complications. We
have to live it and doing partial work (because of UI) is not a good idea.>>
Not sure if we are discussing the same thing here. My question was around the
need to have exceptions for deny and allow. If it indeed going to be
complicated, then should we not such areas which add more complication.
<<The good thing about Madhan’s design is, it is full backward compatibility
and it is optional to use deny and even exception. And it provides a lot of
flexibility for expert users to model their policies.>>
I think we should provide both deny and exceptions policies as optional. The
exceptions should be turned off by default and should not appear in the UI.
Users who really need exceptions can turn it on.
On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai
<[email protected]<mailto:[email protected]>> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the
>resource while anyone else in group "interns" is denied to access the
>resource. The policy creation in the UI is much simpler than having to specify
>"Scott" as an exception to Deny and then specifying "Scott" for Allow.
I think, Madhan’s use case is not exactly the same. In his case, he wants to
“Deny” all interns at a high level, but not necessary want to Allow (globally)
“interns” in this policy (which could be bad in some cases). Scott’s
permissions to any sub resources might be managed by other policies.
The main thing here is:
1. When we decided to support “Deny”, we know it would add complications. We
have to live it and doing partial work (because of UI) is not a good idea.
2. We have to document the flow. And I think, Madhan’s flow is simple.
* First Deny Policies, except those who are in exception list
* Then Allow Policies, except those who are in the exception list
3. Finally, you are correct. For most users and use cases, you can model the
policies with our existing ones. So very few will use “Deny” and those would
use it, would have to understand how it works. And I don’t think it is so
complex, but they just need know the about 2 rules
4. If we feel the UI is confusing it, then we need to fix the UI. But I
feel, compared some of the other system which support “Deny” and hierarchy,
ours is pretty elegant and scalable solution.
The good thing about Madhan’s design is, it is full backward compatibility and
it is optional to use deny and even exception. And it provides a lot of
flexibility for expert users to model their policies.
Bosco
From: Balaji Ganesan
Reply-To:
<[email protected]<mailto:[email protected]>>
Date: Thursday, October 15, 2015 at 10:32 AM
To: <[email protected]<mailto:[email protected]>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
I think we completely missed my earlier point. My question was around whether
we need to have exceptions in Deny and Allow beyond just Allow or Deny policy
items. I used the MSFT example to see if they have implemented exceptions to
Deny, which I did not find any. I am not suggesting that we ask the users to
order the policies and Ranger follow the order.
We can open a separate thread on whether Ranger needs to use an explicit
hierarchy for policy evaluation or not. For the current discussion, I don't see
a specific use case where we would need a deny or allow exception and not
fulfilled by just allow or deny functionality.
Let us take the use case we discussed earlier. The group "interns" got
denied,but we want to exclude one user ("Scott") from that, I am suggesting to
have the particular user ("Scott") included in the Allow section of policy and
group "interns" included in Deny section of the same policy. Ranger should
evaluate the both Allow and Deny in the same policy and deduce that user
"Scott" can access the resource while anyone else in group "interns" is denied
to access the resource. The policy creation in the UI is much simpler than
having to specify "Scott" as an exception to Deny and then specifying "Scott"
for Allow.
On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj
<[email protected]<mailto:[email protected]>> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users
to setup ACE in specific order – see below.
From
https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed
ACEs; you do not need to explicitly deny access to an object. The exception is
when an ACE allows access to a group and you want to deny access to a member of
the group. To do this, place an access-denied ACE for the user in the DACL
ahead of the access-allowed ACE for the group. Note that the order of the
ACEs<https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx>
is important because the system reads the ACEs in sequence until access is
granted or denied. The user's access-denied ACE must appear first; otherwise,
when the system reads the group's access allowed ACE, it will grant access to
the restricted user."
This approach of relying on ordering of policy items (equivalent of ACEs above)
can work for Ranger as well. However Microsoft DACL model evaluates only one
DACL to determine access to an object; compare this with Ranger where there
could be multiple policies that can be evaluated for a resource – for example
to determine access to a finance.invoice.ssn column, Ranger would evaluate
policies that are applicable for finance database, finance.invoice table and
finance.invoice.ssn column. To guarantee consistent authorization result, that
relies on proper ordering of allow/deny policy items in each policy, would
require the policies to be evaluated in a specific order – either
most-specific-to-most-generic (column policy, table policy, database policy)
or most-generic-to most-specific. And it might be very challenging, if not
impossible, to ensure a policy order given our support for wildcard
specification – for example, should a policy for “/finance/*2015/*” be
evaluated before a policy for “/finance/invoice*/*” while determining access
for a file named “/finance/invoice2015/vendor1.txt”?
I think this approach would make the policy authoring very difficult and
confusing, in addition to being not able to support certain scenarios like
“deny at a higher level (like for a database), even if access might be allowed
at a lower level (for a table/column)”, “allow at a higher level, but deny at a
lower level). The current implementation (in tag-policy branch of Ranger) is
much less confusing and offers building blocks that can be used to support more
usecases and can guarantee consistent results for a given set of policies. If
we are after simplicity, the DACL model does not seem to be the right choice..
Thanks,
Madhan
From: Balaji Ganesan
Reply-To:
"[email protected]<mailto:[email protected]>"
Date: Wednesday, October 14, 2015 at 10:31 AM
To: "[email protected]<mailto:[email protected]>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
>> My proposal is to just keep the policy to have only allow and deny with NO
>> exceptions. In your use case, if we allow "finance" group and deny "intern"
>> group, then anyone in intern group would not be allowed while everyone else
>> in finance group will get access. If there is a person in intern group who
>> needs access, then user need to be taken out of the intern group or we need
>> to add only the users who specifically need to be denied.
<<Users who are not comfortable with the idea of “excludes” can continue to use
only allow and deny in the policies. Users who are comfortable with “excludes”
can choose to use it to simplify their policy management.>>
The question here is whether exceptions are really needed in the product. We
need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond
simply access grant and access deny policy line items. As an example, I am
looking at MSFT for how they have have implemented deny-access
https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
The use case MSFT is trying to solve it denying access to specific users who
are part of the group which is provided access. I think most of the deny-access
use cases would be for specific users, not for large sets of groups.
On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj
<[email protected]<mailto:[email protected]>> wrote:
>> For argument sakes, finance could be the higher level group and intern could
>> be a sub group in a hierarchy. There could be n number of sub groups. We
>> could provide access by adding n-1 groups to the policy. We can argue
>> whether n<10 or n>50, depending on the answer it would make sense to add
>> all groups which needs access or specify deny groups which don't need access
The groups may not fall into nice hierarchies – for example, interns group
might consist of users from various orgs in a company (not just from finance
group). In such cases, the only choice is to setup Ranger policies with
individual users (and not groups).
>> My proposal is to just keep the policy to have only allow and deny with NO
>> exceptions. In your use case, if we allow "finance" group and deny "intern"
>> group, then anyone in intern group would not be allowed while everyone else
>> in finance group will get access. If there is a person in intern group who
>> needs access, then user need to be taken out of the intern group or we need
>> to add only the users who specifically need to be denied.
Users who are not comfortable with the idea of “excludes” can continue to use
only allow and deny in the policies. Users who are comfortable with “excludes”
can choose to use it to simplify their policy management.
Thanks,
Madhan
From: Balaji Ganesan
Reply-To:
"[email protected]<mailto:[email protected]>"
Date: Tuesday, October 13, 2015 at 10:24 PM
To: "[email protected]<mailto:[email protected]>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
<<Are you suggesting that the security admins create policies that list
individual users, instead of using groups? Wouldn’t that make security
administration more painful, in a reasonably large organization? For every
change in users role (or group) or an employee joining/leaving the org, all
security policies have to be reviewed and updated.. Many ACLs support both
users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be
a sub group in a hierarchy. There could be n number of sub groups. We could
provide access by adding n-1 groups to the policy. We can argue whether n<10 or
n>50, depending on the answer it would make sense to add all groups which
needs access or specify deny groups which don't need access
My proposal is to just keep the policy to have only allow and deny with NO
exceptions. In your use case, if we allow "finance" group and deny "intern"
group, then anyone in intern group would not be allowed while everyone else in
finance group will get access. If there is a person in intern group who needs
access, then user need to be taken out of the intern group or we need to add
only the users who specifically need to be denied.
On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj
<[email protected]<mailto:[email protected]>> wrote:
Balaji,
>> why cannot user clearly specific users or groups that would need specific
>> access. Why is there a need to give access to finance group as a whole when
>> we know there is a subset of user who do not need access to the the finance
>> database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list
individual users, instead of using groups? Wouldn’t that make security
administration more painful, in a reasonably large organization? For every
change in users role (or group) or an employee joining/leaving the org, all
security policies have to be reviewed and updated.. Many ACLs support both
users and groups to alleviate this issue.
From: Balaji Ganesan
Reply-To:
"[email protected]<mailto:[email protected]>"
Date: Tuesday, October 13, 2015 at 9:01 PM
To: "[email protected]<mailto:[email protected]>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
Madhan, Fantastic job in putting together in the wiki. Thank you.
We clearly need to show case use cases for deny exclude and allow exclude. In
my opinion it is very confusing to user to construct such a policy
<<Let’s say one of the users, scott who is in interns and finance groups, works
on an assignment that requires select access to finance database. To enable
this access, the authorization policy for the database should be updated by
adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception
for Scott. First of all, why cannot user clearly specific users or groups that
would need specific access. Why is there a need to give access to finance group
as a whole when we know there is a subset of user who do not need access to the
the finance database? or clearly specify users who need to be denied? It is
confusing to think about exceptions for deny, deny, exceptions for allow and
allow.
On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj
<[email protected]<mailto:[email protected]>> wrote:
Bosco,
Thanks for the review and comments. The
wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
has been updated to address your comments. Please review.
Thanks,
Madhan
From: Don Bosco Durai
Reply-To:
"[email protected]<mailto:[email protected]>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "[email protected]<mailto:[email protected]>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
Madhan, thanks for putting this document together. It is looking good.
Can I make a few suggestions:
1. Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy
that allows all finance group users to access contents of /finance folder, but
denies access to users in interns group. Users in interns group will be denied
the access even if they are part of finance group.”
2. Can we also add a simple use case of global “Deny”. E.g Deny all users
from “interns” group from accessing table “Employees"
3. The label “Exceptions”, can we make it more explicit. E.g. “Exclude from
Allow Conditions” and “Exclude from Deny Conditions”
4. Probably one small paragraph to explain “Exceptions” will be good. I
think, this is sort of a new concept.
5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create
flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.
Bosco
From: Madhan Neethiraj
Reply-To:
<[email protected]<mailto:[email protected]>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "[email protected]<mailto:[email protected]>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and
exceptions
All,
Apache Ranger policy model enhancement to support deny-conditions and
exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is
available in tag-policy
branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This
enhancement adds the capability to explicitly deny access to resources based on
users/groups, access-types and custom-conditions. It also supports allow/deny
to be specified for a wider group (like employees, public, etc) but exclude
specific users/groups who might be part of the wider groups.
An overview of the implementation, along with few examples is available in
Apache wiki page
here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
Please review.
Thanks,
Madhan
