>I want to know why audits are showing that it is because of hadoop-acl not >ranger-acl? Hafiz, this is a good question and we should probably document it or come with a blog for this.
Only for HDFS and YARN, we support falling back to native permission check if we don’t have corresponding permission in Ranger. So in your case, since there were no permissions in Ranger for “asma” to the folder “/mjd”, we went and checked hadoop-acl. And since even hadoop didn’t have native posix ACL for asma for the folder /mjd, it denied it. Since hadoop was the last one to deny, you saw “hadoop-acl” in the audit record. If in the HDFS level, you had given rwx-rwx-rwx ACLs, then HDFS would have allowed creating the folder and the audit would should that hadoop-acl allowed to create the folder. This also answers yours previous question why we want to make umask=077 and chmod –r 000 to all application folders to be managed by Ranger. So if there are no Ranger policies, then we want to hadoop also to deny. With the recent deny feature, you can explicitly “deny” “asma” or any group from creating/writing. Or you could deny all, but exclude “developer’ and “sadaf” from the deny users. In the future release, I feel, we should provide a way to mark certain folders to be managed exclusively by Ranger. And that will remove a lot of confusion and also make the policy management more predictable. Does it answer your question? Bosco From: Hafiz Mujadid <hafizmujadi...@gmail.com> Reply-To: <user@ranger.incubator.apache.org> Date: Tuesday, December 1, 2015 at 8:59 PM To: <user@ranger.incubator.apache.org> Subject: Re: Group level permission are not working in ranger Hi Bosco! I created a directory /mjd with following permissions drwxr-xr-x - hduser supergroup 0 2015-12-02 09:44 /mjd Then i made a policy with following permissions Datascientist group has one user asma and developer group has one user named haniya and sadaf has no group. So when i run following command HADOOP_USER_NAME=asma hdfs dfs -mkdir /mjd/a1 mkdir: Permission denied: user=asma, access=WRITE, inode="/mjd/a1":hduser:supergroup:drwxr-xr-x And audit of this command is as follow Service Policy IDEvent TimeUserName / TypeResource NameAccess TypeResultAccess EnforcerClient IPEvent Count --12/02/2015 09:46:23 AMasmahdfsRepo /mjd/a1WRITEDeniedhadoop-acl192.168.23.1051 I want to know why audits are showing that it is because of hadoop-acl not ranger-acl? Thanks On Wed, Dec 2, 2015 at 9:37 AM, Don Bosco Durai <bo...@apache.org> wrote: You don’t need to. Since auditing is working, you can check who gave the permission without 000 We recommend giving 000 at HDFS level, because Ranger by default falls back to HDFS permission. So for all folders you want to Ranger to be exclusive, you give as minimal permission as possible. I think, we should also make it configurable in Ranger. Where you can tell Ranger for these folders, it shouldn’t fall back to HDFS. So you don’t have to worry about HDFS level ACLs. The reason you don’t want Ranger to manage everything because there are folders like tmp and user folders which want the system and user to manage themselves. But for application folders like Hive warehouse, you should let Ranger manage it. Bosco From: Hafiz Mujadid <hafizmujadi...@gmail.com> Reply-To: <user@ranger.incubator.apache.org> Date: Tuesday, December 1, 2015 at 1:31 PM To: <user@ranger.incubator.apache.org> Subject: Re: Group level permission are not working in ranger Hi, Bosco, I noticed group level permission works when we set hadoop permissions to 000. I am just curious why it is so ? is it always necessary to set hadoop permissions to 000 for ranger to work? thanks On Mon, Nov 30, 2015 at 10:59 PM, Hafiz Mujadid <hafizmujadi...@gmail.com> wrote: Bosco, I have tried both mysql db and solr as well, only plugin related auditing is being shown On Mon, Nov 30, 2015 at 10:53 PM, Don Bosco Durai <bo...@apache.org> wrote: Yes, you should fix audit first. That will help in debugging these issues also. BTW, are you using Solr or DB? Recommendation is to use Solr. Yesterday, I have uploaded a new package for setting up Solr. It is available as attachment in https://issues.apache.org/jira/browse/RANGER-728. The instructions are in https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5 Give it a try. Thanks Bosco From: Madhan Neethiraj <mneethi...@hortonworks.com> Reply-To: <user@ranger.incubator.apache.org> Date: Monday, November 30, 2015 at 8:57 AM To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> Subject: Re: Group level permission are not working in ranger Hafiz, Few things to check: 1. Do you have another policy in Ranger that allows WRITE access? 2. Can you disable this policy and try mkdir? Fixing the issue with audit will help; audit log will have the details of how the access was allowed (hadoop-acl or ranger-acl; in case of ranger-acl, the policy-ID that determined the access). Madhan From: Hafiz Mujadid <hafizmujadi...@gmail.com> Reply-To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> Date: Monday, November 30, 2015 at 6:16 AM To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> Subject: Re: Group level permission are not working in ranger Bosco, I have followed above steps drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg changed the umask so newly created folder or files have following permissions d---rwxrwx - asma hadoop 0 2015-11-30 19:03 /pg/b i changed the ownership of all folders in hdfs with hduser:hadoop ran the command hdfs dfs -chmod -R 000 /pg but still group level permissions are not working. my audits are not working, i am trying to figure out the issue with audits. i will let you know when audits are available. thanks On Mon, Nov 30, 2015 at 7:13 PM, Hafiz Mujadid <hafizmujadi...@gmail.com> wrote: Bosco, I have followed above steps drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg changed the umask so newly created folder or files have following permissions d---rwxrwx - asma hadoop 0 2015-11-30 19:03 /pg/b i changed the ownership of all folders in hdfs with hduser:hadoop but still group level permissions are not working. my audits are not working, i am trying to figure out the issue with audits. i will let you know when audits are available. thanks On Mon, Nov 30, 2015 at 9:34 AM, Don Bosco Durai <bo...@apache.org> wrote: Can you check Ranger Audits? Also, do couple of things: 1. hdfs dfs -ls /pg (check the HDFS level permissions) 2. In HDFS settngs, set the umask to 700 and restart name node. 3. hdfs dfs -chown hdfs:hdfs /pg 4. hdfs dfs -chmod -R 000 /pg For all user folders, e.g. /app/hive, do #3 and #4 as above. Bosco From: Hafiz Mujadid <hafizmujadi...@gmail.com> Reply-To: <user@ranger.incubator.apache.org> Date: Sunday, November 29, 2015 at 8:29 PM To: <user@ranger.incubator.apache.org> Subject: Re: Group level permission are not working in ranger Yes Bosco, directory is being created. On Mon, Nov 30, 2015 at 2:47 AM, Don Bosco Durai <bo...@apache.org> wrote: What is happening here? Is the directory getting created? Thanks Bosco From: Hafiz Mujadid <hafizmujadi...@gmail.com> Reply-To: <user@ranger.incubator.apache.org> Date: Sunday, November 29, 2015 at 1:44 PM To: <user@ranger.incubator.apache.org> Subject: Group level permission are not working in ranger Hi all I am trying to apply permission on an ldap group but it's not working But when i run following command HADOOP_USER_NAME=asma hdfs dfs -mkdir /pg/b i works successfully what is the issue? ldap users and groups are synced correctly as when i run the command hdfs groups asma it returns correct group asma : datascientist -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID
