When i hit some hdfs command following logs are appended in hadoop log file
2015-12-01 13:24:43,662 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9000, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from 192.168.23.105:56699 Call#0 Retry#0: org.apache.hadoop.security.AccessControlException: Permission denied: user=asma, access=EXECUTE, inode="/pg/m":hduser:hadoop:d--------- 2015-12-01 13:25:17,369 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user hduser 2015-12-01 13:25:17,384 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user hduser 2015-12-01 13:25:17,384 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user hduser 2015-12-01 13:25:17,384 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 192.168.23.105 2015-12-01 13:25:17,385 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs 2015-12-01 13:25:17,385 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 4569 2015-12-01 13:25:17,385 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 57 2015-12-01 13:25:17,399 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 70 2015-12-01 13:25:17,400 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /app/hadoop/tmp/dfs/name/current/edits_inprogress_0000000000000004569 -> /app/hadoop/tmp/dfs/name/current/edits_0000000000000004569-0000000000000004570 2015-12-01 13:25:17,400 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 4571 2015-12-01 13:26:07,565 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9000, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from 192.168.23.105:56703 Call#0 Retry#0: org.apache.hadoop.security.AccessControlException: Permission denied: user=asma, access=EXECUTE, inode="/pg/m":hduser:hadoop:d--------- On Tue, Dec 1, 2015 at 1:18 PM, Hafiz Mujadid <[email protected]> wrote: > Hi Bosco, > > I resolved this issue by placing httpclient jar in ranger-hdfs-plugin/lib > and hadoop/lib and hadoop/share/hadoop/hdfs/lib > > But audits are still not being shown > > latest hadoop log file is attached. Please have a look and suggest me next > step. thanks > > > > On Tue, Dec 1, 2015 at 12:50 PM, Hafiz Mujadid <[email protected]> > wrote: > >> Hi Bosco, >> I checked the hadoop/lib folder there was no httpclient jar file so i >> downloaded and placed 4.2.5 version of httpclient. httpmime is 4.2.5 >> version already. so both the jars are of same version now. But when i start >> hadoop, face following exception as you mentioned >> >> java.lang.NoClassDefFoundError: >> org/apache/http/entity/mime/content/ContentBody >> at >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265) >> at >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:237) >> at >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:217) >> at >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:212) >> at >> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:92) >> at >> org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:56) >> at >> org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:163) >> at >> org.apache.ranger.authorization.hadoop.config.RangerConfiguration.initAudit(RangerConfiguration.java:127) >> at >> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:89) >> at >> org.apache.ranger.authorization.hadoop.RangerHdfsPlugin.init(RangerHdfsAuthorizer.java:400) >> at >> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:84) >> at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1074) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:678) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:664) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488) >> at >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554) >> Caused by: java.lang.ClassNotFoundException: >> org.apache.http.entity.mime.content.ContentBody >> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >> >> >> On Tue, Dec 1, 2015 at 4:04 AM, Don Bosco Durai <[email protected]> wrote: >> >>> I see the following errors: >>> 2015-12-01 02:56:30,065 FATAL >>> org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect to >>> Solr server. URL=[http://192.168.23.105:6083/solr/ranger_audits] >>> java.lang.NoClassDefFoundError: >>> org/apache/http/entity/mime/content/ContentBody >>> at >>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265) >>> >>> I had similar issues previously with incompatible http-client and >>> http-mime jar files. I am not sure whether you are hitting the sample. >>> >>> Can you check the hadoop lib folders for *http*.jar files? Mine >>> is httpclient-4.2.5.jar and we package httpmime-4.2.5.jar with HDFS >>> plugin. >>> >>> Thanks >>> >>> Bosco >>> >>> >>> From: Hafiz Mujadid <[email protected]> >>> Reply-To: <[email protected]> >>> Date: Monday, November 30, 2015 at 2:22 PM >>> To: <[email protected]> >>> >>> Subject: Re: access audit logs are not being shown on ranger ui >>> >>> Hi Bosco, >>> >>> Attached is the content of namenode log file. >>> Solr is working fine, i checked from the web UI >>> Also /var/log/hadoop/hdfs/audit/solr/spool contains following contents: >>> >>> archive >>> index_batch_batch.solr_hdfs.json >>> index_batch_batch.solr_hdfs_closed.json >>> >>> and all of these files and folders are empty >>> >>> >>> Hi Madhan, >>> >>> I have used same URL but still it does not work. >>> >>> >>> >>> >>> >>> On Tue, Dec 1, 2015 at 12:26 AM, Madhan Neethiraj <[email protected]> >>> wrote: >>> >>>> Hafiz, >>>> >>>> Different Solr URLs are specified in configuration of HDFS plugin and >>>> Ranger Admin. Can you please review and update such that they both point to >>>> the same Solr instance? >>>> >>>> properties for hdfs plugin: >>>> >>>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits >>>> <http://192.168.23.105:6083/solr/ranger_audits>* >>>> >>>> >>>> properties in ranger-admin: >>>> >>>> >>>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits >>>> <http://122.129.79.70:6083/solr/ranger_audits>* >>>> >>>> >>>> Madhan >>>> >>>> >>>> From: Don Bosco Durai <[email protected]> >>>> Reply-To: "[email protected]" < >>>> [email protected]> >>>> Date: Monday, November 30, 2015 at 10:48 AM >>>> To: "[email protected]" < >>>> [email protected]> >>>> Subject: Re: access audit logs are not being shown on ranger ui >>>> >>>> The properties seems correct. Best way to test is to : >>>> >>>> 1. Check the HDFS logs. >>>> 2. Check Solr from web interface (use the same url you configured) >>>> 3. See if there are any unprocessed files in >>>> */var/log/hadoop/hdfs/audit/solr/spool* >>>> >>>> Bosco >>>> >>>> From: Hafiz Mujadid <[email protected]> >>>> Reply-To: <[email protected]> >>>> Date: Monday, November 30, 2015 at 7:22 AM >>>> To: <[email protected]> >>>> Subject: access audit logs are not being shown on ranger ui >>>> >>>> hi, >>>> >>>> I have enabled ranger audits in solr, but access audits are not being >>>> shown. >>>> properties for hdfs plugin >>>> *XAAUDIT.SOLR.ENABLE=true* >>>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits >>>> <http://192.168.23.105:6083/solr/ranger_audits>* >>>> *XAAUDIT.SOLR.USER=ranger_solr* >>>> *XAAUDIT.SOLR.PASSWORD=NONE* >>>> *XAAUDIT.SOLR.ZOOKEEPER=NONE* >>>> *XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool* >>>> >>>> properties in ranger-admin >>>> >>>> >>>> *audit_store=solr* >>>> >>>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits >>>> <http://122.129.79.70:6083/solr/ranger_audits>* >>>> *audit_solr_user=ranger_solr* >>>> *audit_solr_password=NONE* >>>> *audit_solr_zookeepers=* >>>> >>>> But no audits logs are shown in ranger-admin >>>> >>>> >>>> how can i check the issue ? >>>> >>>> thanks >>>> >>>> >>> >>> >>> -- >>> Regards: HAFIZ MUJADID >>> >>> >> >> >> -- >> Regards: HAFIZ MUJADID >> > > > > -- > Regards: HAFIZ MUJADID > -- Regards: HAFIZ MUJADID
