Thanks madhan, for the detailed explanation. I am now able to understand the audit logging much better.
Appreciate your help and support. Regards Aruna On 21 Dec 2015 20:57, "Madhan Neethiraj" <mad...@apache.org> wrote: > Aruna, > > >> If user "aruna" does any other action, other than submit-app on the > resource "default". Then the audit log will show the result as "denied". In > this case, should the policy id which is 656 be shown along with the > "denied" result, or the id will be a blank field. > Lets say the action was “write”. In this case, policy #656 would not > “deny” the access. The policy simply was not able to make an authorization > decision – since the attributes of the request (user, group, access-type) > do not match with the policy. In such case, the policy engine will look for > the next policy that matches the request. Following scenario should help > clarify this. > > In addition to 656, consider there is another policy with: “id=657; > resource=*; user=john; access-type=write”; here the resource and > access-type in the policy match the request, but the username does not. > Hence this policy can not make the authorization decision. In this case, > should we expect the audit log to contain both 656 and 657 as policyIds? > Remember, there could be multiple such policies that do not match the > request. > > > >> Under what cases will the audit log show result as "denied" along with > policy id.? > Ranger policy model was recently enhanced to be able to actively deny > access – refer the documentation here > <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>. > Only when such a policy explicitly denies an access, audit log will record > its policyId. > > Hope this helps. > > Madhan > > From: Aruna Sivaram <sivaram.ar...@gmail.com> > Reply-To: "user@ranger.incubator.apache.org" < > user@ranger.incubator.apache.org> > Date: Monday, December 21, 2015 at 5:59 AM > To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> > Subject: Re: Queries on the developement for a new custom plugin > > Madhan, > > let's say i have a policy with following details > > Policy id : 656 > Resource name : default > User : aruna > Access type : submit-app > > If user "aruna" does any other action, other than submit-app on the > resource "default". Then the audit log will show the result as "denied". In > this case, should the policy id which is 656 be shown along with the > "denied" result, or the id will be a blank field. That's my confusion. > > Under what cases will the audit log show result as "denied" along with > policy id.? > > Thanks > Aruna > On 17 Dec 2015 22:30, "Madhan Neethiraj" <mad...@apache.org> wrote: > >> Aruna, >> >> >> However, the policyId still shows blank. Is this the right behavior? >> Ranger populates policyId field in audit log only when that policy makes >> the authorization decision – either allow or deny. In this particular case, >> no policy explicitly allowed or denied the access. The end result is “deny” >> - only because there was no policy to allow the access. Hence the policyId >> is left blank. Hope this helps. To help understand this further, try to >> answer this question: which policyId should be recorded in the audit log in >> this case? Please consider that there could be multiple policies applicable >> for the resource being accessed – like queue=*, queue=test*, queue=t*. >> >> Thanks, >> Madhan >> >> From: Aruna Sivaram <sivaram.ar...@gmail.com> >> Reply-To: "user@ranger.incubator.apache.org" < >> user@ranger.incubator.apache.org> >> Date: Thursday, December 17, 2015 at 12:17 AM >> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> >> Subject: Re: Queries on the developement for a new custom plugin >> >> Madhan, >> >> I was able to fix this issue for the 3rd case. Thing is that there was a >> spelling mismatch between the actionType specified in the servicedef and >> the actionType that is sent through the request. Now, with the change i am >> getting the Result as "Denied" instead of "allowed". However, the policyId >> still shows blank. Is this the right behavior? >> >> I am attaching a screenshot for your reference. >> >> Regards >> Aruna >> >> On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <mad...@apache.org> >> wrote: >> >>> Aruna, >>> >>> >> 2nd case --> if a different user "it3" does a "submit-app" on the >>> "default" queue, then the audit log shows the result as "denied". However, >>> the policy id is blank, is this the right behavior? the isAccessAllowed >>> returns “Denied" >>> >>> Since there was no policy that allowed the access, the request was >>> denied. Hence the policyId is blank. >>> >>> >>> >>> >> 3rd case --> if the user "network3" does a "admin-queue" on the >>> "default" queue, then the audit log shows "allowed", instead of "denied” >>> >>> The audit log clearly says that the access was granted by policyId=656. >>> Can you please send the screenshot of the policy? Another possibility is if >>> your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as >>> shown below); can you please check if this is the case? >>> >>> "accessTypes": >>> >>> [ >>> >>> { >>> >>> "itemId": 1, >>> >>> "name": "SUBMIT_APP", >>> >>> "label": “SUBMIT_APP”, >>> >>> "impliedGrants": >>> >>> [ >>> >>> "ADMIN_QUEUE" >>> >>> ] >>> >>> }, >>> >>> >>> { >>> >>> "itemId": 2, >>> >>> "name": “ADMIN_QUEUE", >>> >>> "label": "ADMIN_QUEUE" >>> >>> } >>> >>> ] >>> >>> >>> Thanks, >>> Madhan >>> >>> From: Aruna Sivaram <sivaram.ar...@gmail.com> >>> Reply-To: "user@ranger.incubator.apache.org" < >>> user@ranger.incubator.apache.org> >>> Date: Monday, December 14, 2015 at 10:47 PM >>> >>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org >>> > >>> Subject: Re: Queries on the developement for a new custom plugin >>> >>> Madhan, >>> >>> Let assume that I have created a custom policy named TestPolicy, Policy >>> Id : 656 for a user called "network3" giving "submit-app" permissions for >>> a queue called "default". >>> >>> 1st case --> , if the same user "network3" does a "submit-app" on the >>> "default" queue, then the audit log shows the correct result with the >>> Result as "Allowed", the isAccessAllowed method returns "Allowed" >>> >>> 2nd case --> if a different user "it3" does a "submit-app" on the >>> "default" queue, then the audit log shows the result as "denied". However, >>> the policy id is blank, is this the right behavior? the isAccessAllowed >>> returns "Denied" >>> >>> 3rd case --> if the user "network3" does a "admin-queue" on the >>> "default" queue, then the audit log shows "allowed", instead of "denied" (I >>> have created a policy giving only "submit-app" permissions on "default" >>> queue to network3 user). In such a scenario, what should be done? Currently >>> my custom authoriser passes the RangerAccessRequestImpl to the >>> isAccessAllowed Method. >>> and following are the values of the RangerAccessRequestImpl object in >>> this scenario. >>> >>> RangerAccessRequestImpl request = new >>> RangerAccessRequestImpl(); >>> RangerAccessResourceImpl resource = new >>> RangerAccessResourceImpl(); >>> resource.setValue("queue", default); >>> request.setResource(resource); >>> request.setAccessType("admin-queue"); >>> request.setUser("network3"); >>> request.setAccessTime(new Date()); >>> >>> request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate()); >>> request.setAction("admin-queue"); >>> request.setClientIPAddress("10.0.2.15"); >>> RangerAccessResult result = plugin.isAccessAllowed(request); >>> return result == null ? false : result.getIsAllowed(); >>> In this scenario, the isAccessAllowed returns "Allowed", ideally it >>> should return "denied". >>> >>> >>> I have attached the screenshot for your reference, which highlights the >>> 3 cases that i have pointed out. >>> >>> Regards >>> Aruna >>> >>> On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <mad...@apache.org> >>> wrote: >>> >>>> Aruna, >>>> >>>> >> However, in cases, where a user cannot access a resource, the audit >>>> UI (access tab) should show "denied". However, in my case, it stills shows >>>> as "allowed" instead of "denied". It would be helpful if you could tell me >>>> what needs to be done in such a case. I am invoking the >>>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me >>>> know what needs to be passed to this API when a user is not allowed to >>>> access a particular resource. >>>> >>>> 1. Audit log (in access tab) should include the ID of the policy that >>>> allowed the access. Can you please check? >>>> 2. What was the return from isAccessAllowed(request)? Allow or Deny? >>>> >>>> If above does not help to identify the issue, can you please send the >>>> return value from isAccessAllowed(request) and the screenshot of the >>>> corresponding audit log? >>>> >>>> Thanks, >>>> Madhan >>>> >>>> From: Aruna Sivaram <sivaram.ar...@gmail.com> >>>> Reply-To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Date: Monday, December 14, 2015 at 7:07 AM >>>> >>>> To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Subject: Re: Queries on the developement for a new custom plugin >>>> >>>> Bosco, >>>> >>>> Thanks for a very detailed explanation, it has given me a very good >>>> perspective on how ranger works/can be used. >>>> >>>> Our use case is that we plan to use apache ranger for authorization of >>>> a home grown application that is non hadoop in nature. As we are using >>>> ranger for the authorization for all the hadoop based components, we plan >>>> to use ranger for all custom based applications in order to ensure >>>> consistency as well as use the auditing / GUI and various other features of >>>> ranger. >>>> >>>> Good news is that, I was able to write a custom plugin & custom >>>> authoriser in ranger for our service and it is also able to sync >>>> successfully (as seen in the plugin tab of the audit GUI). The audit UI >>>> (access tab) is also showing the details of the policy that has been >>>> accessed. >>>> >>>> However, in cases, where a user cannot access a resource, the audit UI >>>> (access tab) should show "denied". However, in my case, it stills shows as >>>> "allowed" instead of "denied". It would be helpful if you could tell me >>>> what needs to be done in such a case. I am invoking the >>>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me >>>> know what needs to be passed to this API when a user is not allowed to >>>> access a particular resource. >>>> >>>> >>>> Thank you once again. >>>> >>>> Regards >>>> Aruna >>>> >>>> >>>> On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org> >>>> wrote: >>>> >>>>> Aruna, can you give more detail on what you are trying to achieve? >>>>> >>>>> I was searching for integration design diagram, but couldn’t find one. >>>>> We will work on creating one. In the meanwhile, here is the high level. >>>>> >>>>> >>>>> 1. Ranger plugins run within the component process. >>>>> 2. It gives a light java library, which does the following: >>>>> 1. Provides method to check access. >>>>> RangerBasePlugin.isAccessAllow() (explicit, you have to call it) >>>>> 2. Pulls policies from Ranger Admin (implicit) >>>>> 3. Does auditing (implicit) >>>>> 3. Ranger community works closely with the Hadoop component >>>>> community for writing plugins for Hadoop components. Currently, there >>>>> are >>>>> close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, >>>>> Kafka, >>>>> Solr, YARN, Storm, Knox and KMS). We are working with the other >>>>> communities >>>>> to support more. >>>>> 4. The framework is generic and so you can use Ranger to provide >>>>> access control to your home grown application also. The wiki page >>>>> >>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 >>>>> gives >>>>> how to implement custom stack. >>>>> 5. Ranger design philosophy is not to change the authorization of >>>>> the component, but have the component define an interface and abstract >>>>> all >>>>> the actions the component supports. This way the components are free to >>>>> support any actions they want to and evolve them over the period of >>>>> time. >>>>> This also help other security providers to provide similar >>>>> implementation. >>>>> >>>>> So if you are planning to write a custom plugin, I assume you are >>>>> having your app and you want to use Ranger to provide the access control >>>>> and audit. And if that is the case, some of the questions you have asked >>>>> applies mostly on your side. >>>>> >>>>> 1. log4j: Since Ranger plugin is embedded within the process, it >>>>> uses the component’s logging framework. So out here, it will be yours >>>>> 2. If you want to use Ranger plugin, first you need to create an >>>>> interface in your application for authorization. We recommend you >>>>> provide >>>>> the default/native simple implementation. Ranger will implement the >>>>> same >>>>> interface, but Ranger implementation will use the policies from Ranger >>>>> Admin. You can review the following sample implementations: >>>>> >>>>> https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java >>>>> and >>>>> >>>>> https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java >>>>> 3. Once your interface is defined, then the wiki pages tells how >>>>> to define a service using JSON. Which can be loaded in Ranger Admin. >>>>> After >>>>> that you can create the policies from Ranger Admin UI, the REST APIs >>>>> for >>>>> managing polices are automatically possible, the Ranger Admin audit UI >>>>> will >>>>> start showing the audits, etc. >>>>> >>>>> Please give your use case, so we can guide you better. >>>>> >>>>> Thanks >>>>> >>>>> Bosco >>>>> >>>>> >>>>> >>>>> >>>>> From: Ramesh Mani <rm...@hortonworks.com> >>>>> Reply-To: <user@ranger.incubator.apache.org> >>>>> Date: Thursday, December 10, 2015 at 11:12 PM >>>>> >>>>> To: "user@ranger.incubator.apache.org" < >>>>> user@ranger.incubator.apache.org> >>>>> Subject: Re: Queries on the developement for a new custom plugin >>>>> >>>>> Please find the answer below. >>>>> >>>>> From: Aruna Sivaram <sivaram.ar...@gmail.com> >>>>> Reply-To: "user@ranger.incubator.apache.org" < >>>>> user@ranger.incubator.apache.org> >>>>> Date: Thursday, December 10, 2015 at 8:34 PM >>>>> To: "user@ranger.incubator.apache.org" < >>>>> user@ranger.incubator.apache.org> >>>>> Subject: Re: Queries on the developement for a new custom plugin >>>>> >>>>> Thanks for your quick response.Please find my queries inline. >>>>> >>>>> >>>>> On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> >>>>> wrote: >>>>> >>>>> If you have created a CustomService, and if you want to authorize the >>>>>> access of components/resources in your CustomService, then you need to >>>>>> have >>>>>> default authorizer in your CustomService to do authorization, which you >>>>>> will extend in your Ranger custom plugin and will be called when your >>>>>> CustomService needs authorization check. >>>>>> >>>>> >>>>> [*Aruna] : It would be great if you could provide a snippet/example >>>>> where the authoriser code present in the plugin is invoked by the existing >>>>> services (hive/hdfs/storm/kafka) for authorisation. This would give me a >>>>> better picture as to how it exactly works.* >>>>> [RM] Each service dictates how to the register the custom authorizer >>>>> with it. So you need to refer the respective service’s authorization >>>>> mechanism. >>>>> >>>>> >>>>>> For logging, as you had seen it uses log4j you can have log4j >>>>>> appender in the log4.properties and get the log, all hadoop components >>>>>> have >>>>>> their log4j properties file where it specify the location it puts the >>>>>> log. >>>>>> By default it is /var/log/hadoop/ for hadoop. >>>>>> >>>>> >>>>> *[Aruna] : there are many log4j.properties in the ranger code base and >>>>> there is no specific log4j.properties for each of the plugins. Hence, >>>>> which >>>>> log4j.properties do i need to modify. My aim is to see all the logs >>>>> generated by my custom plugin. This would enable me to debug the code >>>>> better.* >>>>> [RM] Here also each service provides it log4j.properties file and that >>>>> is where you define appender for ranger also. >>>>> >>>>> [Aruna ] : The other thing i wanted to know was how does one start >>>>> the plugin or rather initiate the plugin which polls for the policies. I >>>>> see that the enable-<service>-plugin.sh scripts sets the environment and >>>>> copies property files to the right locations, but i dont see where the >>>>> authoriser is instantiated in order to invoke the init() method of the >>>>> RangerbasePlugin. >>>>> [RM] This is already in the wiki page. >>>>> >>>>> >>>>> >>>>>> Thanks for all your help >>>>>> >>>>> >>>>> Aruna >>>>> >>>>>> >>>>>> From: Aruna Sivaram <sivaram.ar...@gmail.com> >>>>>> Reply-To: "user@ranger.incubator.apache.org" < >>>>>> user@ranger.incubator.apache.org> >>>>>> Date: Wednesday, December 9, 2015 at 9:45 PM >>>>>> To: "user@ranger.incubator.apache.org" < >>>>>> user@ranger.incubator.apache.org> >>>>>> Subject: Queries on the developement for a new custom plugin >>>>>> >>>>>> I am using ranger 0.5 for the access control. We are planning to >>>>>> develop a custom plugin which we plan to integrate with the ranger >>>>>> framework. This custom plugin will be used for access control of our >>>>>> components. In order to explore this possibility, i have written a custom >>>>>> plugin as per the example given in the link >>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 >>>>>> >>>>>> Since this is a custom plugin, my authoriser class >>>>>> (CustomServiceAuthoriser), will not be extending any of the hadoop >>>>>> security >>>>>> classes (Eg in case of storm or hive, the RangerStormAuthoriser >>>>>> implements >>>>>> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. >>>>>> I >>>>>> have created the repository and policies for the same through the rest >>>>>> web >>>>>> service. >>>>>> >>>>>> >>>>>> I have written the custom service authoriser as per the example and >>>>>> deployed the same on the sandbox. I have a service named CustomService. >>>>>> What i wanted to know is how the customservice will communicate with my >>>>>> customserviceauthoriser which is contained in my plugin. Currently i dont >>>>>> find any documentation which talks about the mode of communication or >>>>>> rather how the plugin class will be invoked by the service. >>>>>> >>>>>> I am relatively new to ranger so may be I am missing something ? >>>>>> >>>>>> Also, i would like to know the location of the log where each of the >>>>>> plugin classes would be logging. This will help us in debugging the >>>>>> flow. I >>>>>> see a lot of log statements in the ranger plugin code base but am unable >>>>>> to >>>>>> find the location of the logs. >>>>>> >>>>>> Secondly, can ranger be used to develop custom plugins for access >>>>>> control of non hadoop components? >>>>>> >>>>>> Any help from your end would be appreciated >>>>>> >>>>>> -- >>>>>> Regards >>>>>> Aruna Sivaram >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards >>>>> Aruna Sivaram >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards >>>> Aruna Sivaram >>>> >>> >>> >>> >>> -- >>> Regards >>> Aruna Sivaram >>> >> >> >> >> -- >> Regards >> Aruna Sivaram >> >
