Re: access audit logs are not being shown on ranger ui

Hi Bosco, I checked the hadoop/lib folder there was no httpclient jar file so i downloaded and placed 4.2.5 version of httpclient. httpmime is 4.2.5 version already. so both the jars are of same version now. But when i start hadoop, face following exception as you mentioned java.lang.NoClassDefFou

Re: access audit logs are not being shown on ranger ui

I see the following errors: 2015-12-01 02:56:30,065 FATAL org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect to Solr server. URL=[http://192.168.23.105:6083/solr/ranger_audits] java.lang.NoClassDefFoundError: org/apache/http/entity/mime/content/ContentBody at org.apache.solr

Re: hdfs plugin enable issue

Bosco, For now I don’t see any change, what I see in Hafiz manual installation is that classpath hadoop/share/hdfs/lib is the one where the hadoop is looking for. Action item here would be do a manual install using apache bits on a raw hadoop cluster and test it. Internally, I tested it with

Re: hdfs plugin enable issue

Great. Thanks. Can we also update the JIRA with the findings and workarounds. Ramesh, do we need to do any changes from our side? Thanks Bosco From: Hafiz Mujadid Reply-To: Date: Monday, November 30, 2015 at 10:47 AM To: Subject: Re: hdfs plugin enable issue yes it worked, name node i

Re: access audit logs are not being shown on ranger ui

Hafiz, Different Solr URLs are specified in configuration of HDFS plugin and Ranger Admin. Can you please review and update such that they both point to the same Solr instance? properties for hdfs plugin: XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits properties in ranger-admin

Re: hdfs plugin enable issue

of course we can whenever you wan here is my skypeid hafizmujadid On Mon, Nov 30, 2015 at 11:47 PM, Hafiz Mujadid wrote: > yes it worked, name node is started > > On Mon, Nov 30, 2015 at 11:41 PM, Ramesh Mani > wrote: > >> Hafiz! >> >> Please try placing it in hadoop/share/hdfs/lib and try agai

Re: hdfs plugin enable issue

yes it worked, name node is started On Mon, Nov 30, 2015 at 11:41 PM, Ramesh Mani wrote: > Hafiz! > > Please try placing it in hadoop/share/hdfs/lib and try again. > > Thanks, > Ramesh > From: Hafiz Mujadid > Reply-To: "user@ranger.incubator.apache.org" < > user@ranger.incubator.apache.org> > D

Re: access audit logs are not being shown on ranger ui

The properties seems correct. Best way to test is to : Check the HDFS logs. Check Solr from web interface (use the same url you configured) See if there are any unprocessed files in /var/log/hadoop/hdfs/audit/solr/spool Bosco From: Hafiz Mujadid Reply-To: Date: Monday, November 30, 2015 at

Re: hdfs plugin enable issue

Hafiz! Please try placing it in hadoop/share/hdfs/lib and try again. Thanks, Ramesh From: Hafiz Mujadid mailto:hafizmujadi...@gmail.com>> Reply-To: "user@ranger.incubator.apache.org" mailto:user@ranger.incubator.apache.org>> Date: Monday, November 30, 20

Re: hdfs plugin enable issue

Bosco, We haven’t tried manual install using the apache bits, I see that the issue Hafiz facing is that more related to how the manual installation put the plugin jars in hadoop lib path. So if ranger-hdfs-plugin-impl is copied to hadoop/lib, it namenode should come up without issue. I shall

Re: hdfs plugin enable issue

Ramesh! I tried by copying ranger-hdfs-plugin-impl to hadoop/lib but still same issue On Mon, Nov 30, 2015 at 10:58 PM, Hafiz Mujadid wrote: > Yes I am using manual install > > On Mon, Nov 30, 2015 at 10:56 PM, Don Bosco Durai > wrote: > >> Ramesh, Hafiz might be using manual install. Have we

Re: Group level permission are not working in ranger

Bosco, I have tried both mysql db and solr as well, only plugin related auditing is being shown On Mon, Nov 30, 2015 at 10:53 PM, Don Bosco Durai wrote: > Yes, you should fix audit first. That will help in debugging these issues > also. > > BTW, are you using Solr or DB? > > Recommendation is to

Re: hdfs plugin enable issue

Yes I am using manual install On Mon, Nov 30, 2015 at 10:56 PM, Don Bosco Durai wrote: > Ramesh, Hafiz might be using manual install. Have we tried that with the > recent code? We selectively pick jars to be packaged in the components. > > Thanks > > Bosco > > > From: Ramesh Mani > Reply-To: >

Re: hdfs plugin enable issue

Ramesh, Hafiz might be using manual install. Have we tried that with the recent code? We selectively pick jars to be packaged in the components. Thanks Bosco From: Ramesh Mani Reply-To: Date: Monday, November 30, 2015 at 9:53 AM To: "user@ranger.incubator.apache.org" Subject: Re: hdfs

Re: hdfs plugin enable issue

Hafiz, Under lib directory there is a folder ranger-hdfs-plugin-impl, please copy this folder to hadoop/lib and try? Regards, Ramesh From: Hafiz Mujadid mailto:hafizmujadi...@gmail.com>> Reply-To: "user@ranger.incubator.apache.org" mailto:user@ranger.

Re: Group level permission are not working in ranger

Yes, you should fix audit first. That will help in debugging these issues also. BTW, are you using Solr or DB? Recommendation is to use Solr. Yesterday, I have uploaded a new package for setting up Solr. It is available as attachment in https://issues.apache.org/jira/browse/RANGER-728. The ins

Re: Group level permission are not working in ranger

Sorry, my mistake, you have to set the umask as 077 and restart name node. By default, the HDFS umask is 022, which means group and others have rx permissions. So regardless what you set in Ranger, all users will have read access. So you need to change the mask to 077, so by default group and ot

Re: Group level permission are not working in ranger

Hafiz, Few things to check: 1. Do you have another policy in Ranger that allows WRITE access? 2. Can you disable this policy and try mkdir? Fixing the issue with audit will help; audit log will have the details of how the access was allowed (hadoop-acl or ranger-acl; in case of ranger-acl, the

Re: hdfs plugin enable issue

Following are the contents Ramesh! 1. isable-hdfs-plugin.sh 2. install 3. lib 4. upgrade-hdfs-plugin.sh 5. enable-hdfs-plugin.sh 6. install.properties 7. ranger_credential_helper.py 8. upgrade-plugin.py On Mon, Nov 30, 2015 at 9:39 PM, Hafiz Mujadid wrote: > in jira i h

Re: hdfs plugin enable issue

in jira i have attached install.properties, which files else you need? On Mon, Nov 30, 2015 at 9:37 PM, Ramesh Mani wrote: > Hafiz, > > Can I get the details on the files where you untarred the plugging tar, it > should be like ranger-hdfs-plugin? > > This issue seems to be that installation

Re: hdfs plugin enable issue

Hafiz, Can I get the details on the files where you untarred the plugging tar, it should be like ranger-hdfs-plugin? This issue seems to be that installation is not completely correct. Regards, Ramesh From: Hafiz Mujadid mailto:hafizmujadi...@gmail.com>> Reply-To: "user@ranger.incubator.apac

access audit logs are not being shown on ranger ui

hi, I have enabled ranger audits in solr, but access audits are not being shown. properties for hdfs plugin *XAAUDIT.SOLR.ENABLE=true* *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits * *XAAUDIT.SOLR.USER=ranger_solr* *XAAUDIT.SOLR.PASS

Re: hdfs plugin enable issue

Ramesh, when i removed the following jars from the hadoop/share/hdfs/lib and try enabling *ranger-hdfs-plugin-shim-0.5.0.jar ranger-plugin-classloader-0.5.0.jar* I faced following exception *java.lang.ClassNotFoundException: Class org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer*

Re: hdfs plugin enable issue

Bosco, Here is the jira against this issue RANGER-752 thanks On Mon, Nov 30, 2015 at 6:21 PM, Hafiz Mujadid wrote: > Ramesh, > > when i removed the following jars from the hadoop/share/hdfs/lib and try > enabling > > > *ranger-hdfs-plugin-sh

Re: Group level permission are not working in ranger

Bosco, I have followed above steps drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg changed the umask so newly created folder or files have following permissions d---rwxrwx - asma hadoop 0 2015-11-30 19:03 /pg/b i changed the ownership of all folders in hdfs with hduser:hado

Re: Group level permission are not working in ranger

Bosco, By the way, i could not understand the reason to perform above steps, can you explain them? thanks On Mon, Nov 30, 2015 at 7:16 PM, Hafiz Mujadid wrote: > Bosco, > > I have followed above steps > >1. drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg >2. changed the u

Re: Group level permission are not working in ranger

Bosco, I have followed above steps 1. drwxr-xr-x - hduser hadoop 0 2015-11-30 18:49 /pg 2. changed the umask so newly created folder or files have following permissions d---rwxrwx - asma hadoop 0 2015-11-30 19:03 /pg/b 3. i changed the ownership of all folders