Hi Bosco, Thanks for your response, I am testing new feature of ranger Deny,Allow. will send you my findings in short.
Thanks On Thu, Dec 3, 2015 at 10:40 PM, Don Bosco Durai <bo...@apache.org> wrote: > >I want to know why audits are showing that it is because of hadoop-acl > not ranger-acl? > Hafiz, this is a good question and we should probably document it or come > with a blog for this. > > Only for HDFS and YARN, we support falling back to native permission check > if we don’t have corresponding permission in Ranger. So in your case, since > there were no permissions in Ranger for “asma” to the folder “/mjd”, we > went and checked hadoop-acl. And since even hadoop didn’t have native posix > ACL for asma for the folder /mjd, it denied it. Since hadoop was the last > one to deny, you saw “hadoop-acl” in the audit record. If in the HDFS > level, you had given rwx-rwx-rwx ACLs, then HDFS would have allowed > creating the folder and the audit would should that hadoop-acl allowed to > create the folder. > > This also answers yours previous question why we want to make umask=077 > and chmod –r 000 to all application folders to be managed by Ranger. So if > there are no Ranger policies, then we want to hadoop also to deny. > > With the recent deny feature, you can explicitly “deny” “asma” or any > group from creating/writing. Or you could deny all, but exclude “developer’ > and “sadaf” from the deny users. > > In the future release, I feel, we should provide a way to mark certain > folders to be managed exclusively by Ranger. And that will remove a lot of > confusion and also make the policy management more predictable. > > Does it answer your question? > > Bosco > > > From: Hafiz Mujadid <hafizmujadi...@gmail.com> > Reply-To: <user@ranger.incubator.apache.org> > Date: Tuesday, December 1, 2015 at 8:59 PM > > To: <user@ranger.incubator.apache.org> > Subject: Re: Group level permission are not working in ranger > > Hi Bosco! > > I created a directory /mjd with following permissions > *drwxr-xr-x - hduser supergroup 0 2015-12-02 09:44 /mjd* > > Then i made a policy with following permissions > [image: Inline image 1] > Datascientist group has one user asma and developer group has one user > named haniya and sadaf has no group. > > So when i run following command > *HADOOP_USER_NAME=asma hdfs dfs -mkdir /mjd/a1* > *mkdir: Permission denied: user=asma, access=WRITE, > inode="/mjd/a1":hduser:supergroup:drwxr-xr-x* > > > > *And audit of this command is as follow*ServicePolicy IDEvent TimeUserName > / TypeResource NameAccess TypeResultAccess EnforcerClient IPEvent > Count--12/02/2015 > 09:46:23 AMasma > hdfsRepo > /mjd/a1WRITEDeniedhadoop-acl192.168.23.1051 > I want to know why audits are showing that it is because of hadoop-acl not > ranger-acl? > > Thanks > > > > > > On Wed, Dec 2, 2015 at 9:37 AM, Don Bosco Durai <bo...@apache.org> wrote: > >> You don’t need to. Since auditing is working, you can check who gave the >> permission without 000 >> >> We recommend giving 000 at HDFS level, because Ranger by default falls >> back to HDFS permission. So for all folders you want to Ranger to be >> exclusive, you give as minimal permission as possible. >> >> I think, we should also make it configurable in Ranger. Where you can >> tell Ranger for these folders, it shouldn’t fall back to HDFS. So you don’t >> have to worry about HDFS level ACLs. >> >> The reason you don’t want Ranger to manage everything because there are >> folders like tmp and user folders which want the system and user to manage >> themselves. But for application folders like Hive warehouse, you should let >> Ranger manage it. >> >> Bosco >> >> From: Hafiz Mujadid <hafizmujadi...@gmail.com> >> Reply-To: <user@ranger.incubator.apache.org> >> Date: Tuesday, December 1, 2015 at 1:31 PM >> >> To: <user@ranger.incubator.apache.org> >> Subject: Re: Group level permission are not working in ranger >> >> Hi, >> >> Bosco, I noticed group level permission works when we set hadoop >> permissions to 000. I am just curious why it is so ? >> >> is it always necessary to set hadoop permissions to 000 for ranger to >> work? >> >> thanks >> >> On Mon, Nov 30, 2015 at 10:59 PM, Hafiz Mujadid <hafizmujadi...@gmail.com >> > wrote: >> >>> Bosco, I have tried both mysql db and solr as well, only plugin related >>> auditing is being shown >>> >>> On Mon, Nov 30, 2015 at 10:53 PM, Don Bosco Durai <bo...@apache.org> >>> wrote: >>> >>>> Yes, you should fix audit first. That will help in debugging these >>>> issues also. >>>> >>>> BTW, are you using Solr or DB? >>>> >>>> Recommendation is to use Solr. Yesterday, I have uploaded a new package >>>> for setting up Solr. It is available as attachment in >>>> https://issues.apache.org/jira/browse/RANGER-728. The instructions are >>>> in >>>> https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 >>>> >>>> Give it a try. >>>> >>>> Thanks >>>> >>>> Bosco >>>> >>>> >>>> From: Madhan Neethiraj <mneethi...@hortonworks.com> >>>> Reply-To: <user@ranger.incubator.apache.org> >>>> Date: Monday, November 30, 2015 at 8:57 AM >>>> >>>> To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Subject: Re: Group level permission are not working in ranger >>>> >>>> Hafiz, >>>> >>>> Few things to check: >>>> 1. Do you have another policy in Ranger that allows WRITE access? >>>> 2. Can you disable this policy and try mkdir? >>>> >>>> Fixing the issue with audit will help; audit log will have the details >>>> of how the access was allowed (hadoop-acl or ranger-acl; in case of >>>> ranger-acl, the policy-ID that determined the access). >>>> >>>> Madhan >>>> >>>> From: Hafiz Mujadid <hafizmujadi...@gmail.com> >>>> Reply-To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Date: Monday, November 30, 2015 at 6:16 AM >>>> To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Subject: Re: Group level permission are not working in ranger >>>> >>>> Bosco, >>>> >>>> I have followed above steps >>>> >>>> 1. drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg >>>> 2. changed the umask so newly created folder or files have >>>> following permissions >>>> d---rwxrwx - asma hadoop 0 2015-11-30 19:03 /pg/b >>>> 3. i changed the ownership of all folders in hdfs with hduser:hadoop >>>> 4. ran the command hdfs dfs -chmod -R 000 /pg >>>> >>>> >>>> but still group level permissions are not working. >>>> >>>> my audits are not working, i am trying to figure out the issue with >>>> audits. i will let you know when audits are available. >>>> >>>> >>>> thanks >>>> >>>> On Mon, Nov 30, 2015 at 7:13 PM, Hafiz Mujadid < >>>> hafizmujadi...@gmail.com> wrote: >>>> >>>>> Bosco, >>>>> >>>>> I have followed above steps >>>>> drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg >>>>> changed the umask so newly created folder or files have following >>>>> permissions >>>>> d---rwxrwx - asma hadoop 0 2015-11-30 19:03 /pg/b >>>>> i changed the ownership of all folders in hdfs with hduser:hadoop >>>>> >>>>> but still group level permissions are not working. >>>>> >>>>> >>>>> my audits are not working, i am trying to figure out the issue with >>>>> audits. i will let you know when audits are available. >>>>> >>>>> >>>>> thanks >>>>> >>>>> >>>>> On Mon, Nov 30, 2015 at 9:34 AM, Don Bosco Durai <bo...@apache.org> >>>>> wrote: >>>>> >>>>>> Can you check Ranger Audits? >>>>>> >>>>>> Also, do couple of things: >>>>>> 1. hdfs dfs -ls /pg (check the HDFS level permissions) >>>>>> 2. In HDFS settngs, set the umask to 700 and restart name node. >>>>>> 3. hdfs dfs -chown hdfs:hdfs /pg >>>>>> 4. hdfs dfs -chmod -R 000 /pg >>>>>> >>>>>> For all user folders, e.g. /app/hive, do #3 and #4 as above. >>>>>> >>>>>> Bosco >>>>>> >>>>>> >>>>>> From: Hafiz Mujadid <hafizmujadi...@gmail.com> >>>>>> Reply-To: <user@ranger.incubator.apache.org> >>>>>> Date: Sunday, November 29, 2015 at 8:29 PM >>>>>> To: <user@ranger.incubator.apache.org> >>>>>> Subject: Re: Group level permission are not working in ranger >>>>>> >>>>>> Yes Bosco, directory is being created. >>>>>> >>>>>> On Mon, Nov 30, 2015 at 2:47 AM, Don Bosco Durai <bo...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> What is happening here? Is the directory getting created? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Bosco >>>>>>> >>>>>>> >>>>>>> From: Hafiz Mujadid <hafizmujadi...@gmail.com> >>>>>>> Reply-To: <user@ranger.incubator.apache.org> >>>>>>> Date: Sunday, November 29, 2015 at 1:44 PM >>>>>>> To: <user@ranger.incubator.apache.org> >>>>>>> Subject: Group level permission are not working in ranger >>>>>>> >>>>>>> Hi all >>>>>>> >>>>>>> I am trying to apply permission on an ldap group but it's not working >>>>>>> >>>>>>> [image: Inline image 1] >>>>>>> >>>>>>> >>>>>>> But when i run following command >>>>>>> *HADOOP_USER_NAME=asma hdfs dfs -mkdir /pg/b* >>>>>>> >>>>>>> i works successfully >>>>>>> what is the issue? ldap users and groups are synced correctly as >>>>>>> when i run the command *hdfs groups asma* it returns correct >>>>>>> group >>>>>>> asma : datascientist >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards: HAFIZ MUJADID >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards: HAFIZ MUJADID >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards: HAFIZ MUJADID >>>> >>>> >>> >>> >>> -- >>> Regards: HAFIZ MUJADID >>> >> >> >> >> -- >> Regards: HAFIZ MUJADID >> >> > > > -- > Regards: HAFIZ MUJADID > > -- Regards: HAFIZ MUJADID
