Kerberos integration with Ranger

[Trainee Bingo]

Hi All,

Hope all user's are having great experience with Ranger.

I have one question that can we integrate Kerberos with Ranger? Can anyone
explain me the answer in detail?



Cheers

Re: Group level permission are not working in ranger

[Don Bosco Durai]

>I want to know why audits are showing that it is because of hadoop-acl not 
>ranger-acl? 
Hafiz, this is a good question and we should probably document it or come with 
a blog for this.

Only for HDFS and YARN, we support falling back to native permission check if 
we don’t have corresponding permission in Ranger. So in your case, since there 
were no permissions in Ranger for “asma” to the folder “/mjd”, we went and 
checked hadoop-acl. And since even hadoop didn’t have native posix ACL for asma 
for the folder /mjd, it denied it. Since hadoop was the last one to deny, you 
saw “hadoop-acl” in the audit record. If in the HDFS level, you had given 
rwx-rwx-rwx ACLs, then HDFS would have allowed creating the folder and the 
audit would should that hadoop-acl allowed to create the folder.

This also answers yours previous question why we want to make umask=077 and 
chmod –r 000 to all application folders to be managed by Ranger. So if there 
are no Ranger policies, then we want to hadoop also to deny.

With the recent deny feature, you can explicitly “deny” “asma” or any group 
from creating/writing. Or you could deny all, but exclude “developer’ and 
“sadaf” from the deny users. 

In the future release, I feel, we should provide a way to mark certain folders 
to be managed exclusively by Ranger. And that will remove a lot of confusion 
and also make the policy management more predictable.

Does it answer your question?

Bosco


From:  Hafiz Mujadid 
Reply-To:  
Date:  Tuesday, December 1, 2015 at 8:59 PM
To:  
Subject:  Re: Group level permission are not working in ranger

Hi Bosco!

I created a directory /mjd with following permissions
drwxr-xr-x   - hduser supergroup  0 2015-12-02 09:44 /mjd

Then i made a policy with following permissions 
Datascientist group has one user asma and developer group has one user named 
haniya and sadaf has no group.

So when i run following command 
HADOOP_USER_NAME=asma hdfs dfs -mkdir /mjd/a1
mkdir: Permission denied: user=asma, access=WRITE, 
inode="/mjd/a1":hduser:supergroup:drwxr-xr-x

And audit of this command is as follow

Service
Policy IDEvent TimeUserName / TypeResource NameAccess TypeResultAccess 
EnforcerClient IPEvent Count
--12/02/2015 09:46:23 AMasmahdfsRepo
/mjd/a1WRITEDeniedhadoop-acl192.168.23.1051
I want to know why audits are showing that it is because of hadoop-acl not 
ranger-acl? 

Thanks





On Wed, Dec 2, 2015 at 9:37 AM, Don Bosco Durai  wrote:
You don’t need to. Since auditing is working, you can check who gave the 
permission without 000

We recommend giving 000 at HDFS level, because Ranger by default falls back to 
HDFS permission. So for all folders you want to Ranger to be exclusive, you 
give as minimal permission as possible.

I think, we should also make it configurable in Ranger. Where you can tell 
Ranger for these folders, it shouldn’t fall back to HDFS. So you don’t have to 
worry about HDFS level ACLs.

The reason you don’t want Ranger to manage everything because there are folders 
like tmp and user folders which want the system and user to manage themselves. 
But for application folders like Hive warehouse, you should let Ranger manage 
it.

Bosco

From:  Hafiz Mujadid 
Reply-To:  
Date:  Tuesday, December 1, 2015 at 1:31 PM

To:  
Subject:  Re: Group level permission are not working in ranger

Hi,

Bosco, I noticed group level permission works when we set hadoop permissions to 
000. I am just curious why it is so ?

is it always necessary to set hadoop permissions to 000 for ranger to work? 

thanks

On Mon, Nov 30, 2015 at 10:59 PM, Hafiz Mujadid  
wrote:
Bosco, I have tried both mysql db and solr as well, only plugin related 
auditing is being shown

On Mon, Nov 30, 2015 at 10:53 PM, Don Bosco Durai  wrote:
Yes, you should fix audit first. That will help in debugging these issues also.

BTW, are you using Solr or DB?

Recommendation is to use Solr. Yesterday, I have uploaded a new package for 
setting up Solr. It is available as attachment in  
https://issues.apache.org/jira/browse/RANGER-728. The instructions are in 
https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5

Give it a try.

Thanks

Bosco


From:  Madhan Neethiraj 
Reply-To:  
Date:  Monday, November 30, 2015 at 8:57 AM

To:  "user@ranger.incubator.apache.org" 
Subject:  Re: Group level permission are not working in ranger

Hafiz,

Few things to check:
 1. Do you have another policy in Ranger that allows WRITE access?
 2. Can you disable this policy and try mkdir?

Fixing the issue with audit will help; audit log will have the details of how 
the access 

Re: Group level permission are not working in ranger

[Hafiz Mujadid]

Hi Bosco,

Thanks for your response, I am testing new feature of ranger Deny,Allow.
will send you my findings in short.

Thanks

On Thu, Dec 3, 2015 at 10:40 PM, Don Bosco Durai  wrote:

> >I want to know why audits are showing that it is because of hadoop-acl
> not ranger-acl?
> Hafiz, this is a good question and we should probably document it or come
> with a blog for this.
>
> Only for HDFS and YARN, we support falling back to native permission check
> if we don’t have corresponding permission in Ranger. So in your case, since
> there were no permissions in Ranger for “asma” to the folder “/mjd”, we
> went and checked hadoop-acl. And since even hadoop didn’t have native posix
> ACL for asma for the folder /mjd, it denied it. Since hadoop was the last
> one to deny, you saw “hadoop-acl” in the audit record. If in the HDFS
> level, you had given rwx-rwx-rwx ACLs, then HDFS would have allowed
> creating the folder and the audit would should that hadoop-acl allowed to
> create the folder.
>
> This also answers yours previous question why we want to make umask=077
> and chmod –r 000 to all application folders to be managed by Ranger. So if
> there are no Ranger policies, then we want to hadoop also to deny.
>
> With the recent deny feature, you can explicitly “deny” “asma” or any
> group from creating/writing. Or you could deny all, but exclude “developer’
> and “sadaf” from the deny users.
>
> In the future release, I feel, we should provide a way to mark certain
> folders to be managed exclusively by Ranger. And that will remove a lot of
> confusion and also make the policy management more predictable.
>
> Does it answer your question?
>
> Bosco
>
>
> From: Hafiz Mujadid 
> Reply-To: 
> Date: Tuesday, December 1, 2015 at 8:59 PM
>
> To: 
> Subject: Re: Group level permission are not working in ranger
>
> Hi Bosco!
>
> I created a directory /mjd with following permissions
> *drwxr-xr-x   - hduser supergroup  0 2015-12-02 09:44 /mjd*
>
> Then i made a policy with following permissions
> [image: Inline image 1]
> Datascientist group has one user asma and developer group has one user
> named haniya and sadaf has no group.
>
> So when i run following command
> *HADOOP_USER_NAME=asma hdfs dfs -mkdir /mjd/a1*
> *mkdir: Permission denied: user=asma, access=WRITE,
> inode="/mjd/a1":hduser:supergroup:drwxr-xr-x*
>
>
>
> *And audit of this command is as follow*ServicePolicy IDEvent TimeUserName
> / TypeResource NameAccess TypeResultAccess EnforcerClient IPEvent 
> Count--12/02/2015
> 09:46:23 AMasma
> hdfsRepo
> /mjd/a1WRITEDeniedhadoop-acl192.168.23.1051
> I want to know why audits are showing that it is because of hadoop-acl not
> ranger-acl?
>
> Thanks
>
>
>
>
>
> On Wed, Dec 2, 2015 at 9:37 AM, Don Bosco Durai  wrote:
>
>> You don’t need to. Since auditing is working, you can check who gave the
>> permission without 000
>>
>> We recommend giving 000 at HDFS level, because Ranger by default falls
>> back to HDFS permission. So for all folders you want to Ranger to be
>> exclusive, you give as minimal permission as possible.
>>
>> I think, we should also make it configurable in Ranger. Where you can
>> tell Ranger for these folders, it shouldn’t fall back to HDFS. So you don’t
>> have to worry about HDFS level ACLs.
>>
>> The reason you don’t want Ranger to manage everything because there are
>> folders like tmp and user folders which want the system and user to manage
>> themselves. But for application folders like Hive warehouse, you should let
>> Ranger manage it.
>>
>> Bosco
>>
>> From: Hafiz Mujadid 
>> Reply-To: 
>> Date: Tuesday, December 1, 2015 at 1:31 PM
>>
>> To: 
>> Subject: Re: Group level permission are not working in ranger
>>
>> Hi,
>>
>> Bosco, I noticed group level permission works when we set hadoop
>> permissions to 000. I am just curious why it is so ?
>>
>> is it always necessary to set hadoop permissions to 000 for ranger to
>> work?
>>
>> thanks
>>
>> On Mon, Nov 30, 2015 at 10:59 PM, Hafiz Mujadid > > wrote:
>>
>>> Bosco, I have tried both mysql db and solr as well, only plugin related
>>> auditing is being shown
>>>
>>> On Mon, Nov 30, 2015 at 10:53 PM, Don Bosco Durai 
>>> wrote:
>>>
 Yes, you should fix audit first. That will help in debugging these
 issues also.

 BTW, are you using Solr or DB?

 Recommendation is to use Solr. Yesterday, I have uploaded a new package
 for setting up Solr. It is available as attachment in
 https://issues.apache.org/jira/browse/RANGER-728. The instructions are
 in
 https://cwiki.apache.org/confluence/display/RANGER/Install+and+Configure+Solr+for+Ranger+Audits+-+Apache+Ranger+0.5

 Give it a try.

 Thanks

 Bosco